{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "07c5e1da",
   "metadata": {},
   "source": [
    "# Python进阶-ML模型应用4_MultiCollinearity In Linear Regression多重共线性及VIF应用\n",
    "\n",
    " 多重共线性（Multicollinearity）是指线性回归模型中自变量之间存在高度相关性的情况。当两个或多个自变量之间存在较高的相关性时，它们对因变量的解释能力会相互影响，导致模型的不稳定性和不准确性。\n",
    "\n",
    "为了检测多重共线性，可以使用方差膨胀因子（Variance Inflation Factor, VIF）。VIF是一个衡量自变量之间相关性的指标，其值越大表示自变量之间的相关性越强。通常认为，如果一个自变量的VIF大于10，则说明该自变量与其他自变量之间存在严重的多重共线性问题。根据不同的标准，实际上，VIF超过4就已经值得多加注意了。\n",
    "\n",
    "解决多重共线性的方法包括：\n",
    "\n",
    "1. 删除一些自变量：通过逐步回归等方法，选择出对因变量解释能力最强的自变量，并删除其他相关性较强的自变量。\n",
    "2. 增加样本量：增加样本量可以降低自变量之间的相关性，从而减少多重共线性的影响。\n",
    "3. 使用岭回归（Ridge Regression）或Lasso回归等正则化方法：这些方法可以通过向损失函数中添加惩罚项来限制自变量之间的相关性，从而减少多重共线性的影响。\n",
    "4. 主成分分析（Principal Component Analysis, PCA）：PCA可以将原始自变量转换为一组新的无关变量，从而减少多重共线性的影响。 "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c04e4ef7",
   "metadata": {},
   "source": [
    "## 代码示例（一）解决多重共线性问题"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d69f06d",
   "metadata": {},
   "source": [
    "### 1、导入函数库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "74663a68",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries \n",
    "import pandas as pd \n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt \n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cd24c1e",
   "metadata": {},
   "source": [
    "### 2、载入数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "d596ff45",
   "metadata": {},
   "outputs": [],
   "source": [
    "#加载数据集\n",
    "df=pd.read_csv('./kaggle-data/salary_data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bf7b1e0c",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "      <th>Salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.1</td>\n",
       "      <td>39343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>1.3</td>\n",
       "      <td>46205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>1.5</td>\n",
       "      <td>37731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>2.0</td>\n",
       "      <td>43525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>2.2</td>\n",
       "      <td>39891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>2.9</td>\n",
       "      <td>56642</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>3.0</td>\n",
       "      <td>60150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>3.2</td>\n",
       "      <td>54445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.2</td>\n",
       "      <td>64445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>3.7</td>\n",
       "      <td>57189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>3.9</td>\n",
       "      <td>63218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "      <td>55794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>4.0</td>\n",
       "      <td>56957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.1</td>\n",
       "      <td>57081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4.5</td>\n",
       "      <td>61111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>4.9</td>\n",
       "      <td>67938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>31</td>\n",
       "      <td>5.1</td>\n",
       "      <td>66029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.3</td>\n",
       "      <td>83088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>32</td>\n",
       "      <td>5.9</td>\n",
       "      <td>81363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>6.0</td>\n",
       "      <td>93940</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>34</td>\n",
       "      <td>6.8</td>\n",
       "      <td>91738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>7.1</td>\n",
       "      <td>98273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>7.9</td>\n",
       "      <td>101302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "      <td>36</td>\n",
       "      <td>8.2</td>\n",
       "      <td>113812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>8.7</td>\n",
       "      <td>109431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>9.0</td>\n",
       "      <td>105582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.5</td>\n",
       "      <td>116969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>9.6</td>\n",
       "      <td>112635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>10.3</td>\n",
       "      <td>122391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>10.5</td>\n",
       "      <td>121872</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Sex  Age  YearsExperience  Salary\n",
       "0     0   24              1.1   39343\n",
       "1     1   25              1.3   46205\n",
       "2     0   24              1.5   37731\n",
       "3     1   25              2.0   43525\n",
       "4     0   25              2.2   39891\n",
       "5     1   26              2.9   56642\n",
       "6     0   27              3.0   60150\n",
       "7     1   26              3.2   54445\n",
       "8     1   27              3.2   64445\n",
       "9     1   27              3.7   57189\n",
       "10    0   28              3.9   63218\n",
       "11    0   28              4.0   55794\n",
       "12    0   28              4.0   56957\n",
       "13    1   29              4.1   57081\n",
       "14    1   29              4.5   61111\n",
       "15    1   30              4.9   67938\n",
       "16    1   31              5.1   66029\n",
       "17    0   32              5.3   83088\n",
       "18    0   32              5.9   81363\n",
       "19    0   33              6.0   93940\n",
       "20    0   34              6.8   91738\n",
       "21    0   35              7.1   98273\n",
       "22    1   36              7.9  101302\n",
       "23    1   36              8.2  113812\n",
       "24    1   38              8.7  109431\n",
       "25    1   38              9.0  105582\n",
       "26    1   39              9.5  116969\n",
       "27    1   39              9.6  112635\n",
       "28    0   40             10.3  122391\n",
       "29    0   41             10.5  121872"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0b3a47a",
   "metadata": {},
   "source": [
    "### 3、划分特征集和响应集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3b63fe5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#取特征列\n",
    "X=X1=df.iloc[:,:-1]#取全部行，最后一列之前的全部列\n",
    "#取响应列\n",
    "y=df.iloc[:,-1]#全部行，只取最后一列"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b052d2f",
   "metadata": {},
   "source": [
    "### 4、多重共线性判断方法\n",
    "\n",
    "特征之间关联性越强，共线性就越大。所以我们必须找出关联性强的特征进行降关联处理，提高模型泛化能力。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "458d5870",
   "metadata": {},
   "source": [
    "#### （1）关联矩阵法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "52a78651",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>YearsExperience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.050028</td>\n",
       "      <td>0.066731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.050028</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.994212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearsExperience</th>\n",
       "      <td>0.066731</td>\n",
       "      <td>0.994212</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Sex       Age  YearsExperience\n",
       "Sex              1.000000  0.050028         0.066731\n",
       "Age              0.050028  1.000000         0.994212\n",
       "YearsExperience  0.066731  0.994212         1.000000"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#当特征数量较少时，我们可以通过关联矩阵的方式快速找出相关性强的变量\n",
    "X.corr()#查看相关性矩阵——观察各个特征之间的相关性\n",
    "#如果y的值与特征变量x之间的数值越大，则说明y对x的依赖性越强，说明变量y非常符合依赖型变量的特点，is good\n",
    "#如果特征变量x与x之间的相关性越大，则说明特征变量x与x之间独立性更小，is very bad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "052e4d75",
   "metadata": {},
   "source": [
    "由此可见，Age与YearsExperience具有极强的关联性。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c34417",
   "metadata": {},
   "source": [
    "#### （2）绘图法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "5f0940b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.PairGrid at 0x21fb7a06e20>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhUAAAIVCAYAAABm5A1+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABKJUlEQVR4nO3de5wcdZ3v/9dnZhImJJkhhMlkSDIE3KgwkYTsiIrgIignuijxFsRVXH/sBs9PBZfdc2RXfqgs+zt7cXO8rhKVVdRV4qIBWRbhcPGyIBAxhIwRAzFAyGQyiTi5wJBM+nP+6OqhZ9I909Vd1V3V/X4+Hv2Y7rr1p6u/Vf2Zqvp+ytwdERERkUo11ToAERERqQ9KKkRERCQSSipEREQkEkoqREREJBJKKkRERCQSSipEREQkEqlLKpYvX+6AHnrE/SiJ2qMeVXqURO1Rjyo9ikpdUrF79+5ahyAySu1RkkTtUWotdUmFiIiIJFNsSYWZXW9mu8xsU5HxZmafM7PHzWyjmS2LKxYRERGJX0uMy/468AXghiLj3wQsCh6vAr4U/BWpeyMjGfr6h+gfGqarfRo9XW20tDRFNr7S9wfIZJxtew4wsHeYzrZWFs6eTlOTlTw+CSpdT6UsI4r3qJYw31k532+heTIZL7p+ir1HJcO72ls5nIFd+5LbLuHIz94962ieeva5kre3UqZ/6ncHGNj7AgcOjnDCsdM58bj410VsSYW7/8TMFk4wyQXADZ69+cjPzewYM+ty9/64YhJJgpGRDOseeYar1m1i+FCG1ilNXLtiMSuWzKOlpani8ZW+P2R3SLf37eSKtRtGp1m9cinLe+aO7sQnGp8Ela6nUpYRxXtUS5jvrJzvt9A8171vGYP7DhZcP01NVvA9zju5kzs2D5Q1fNbRU7n4NSfw2bu2JLZdwpHr6oTZ0/jIOYvGrKeJtrdSpr/7sQG2DOyv+rqoZaufBzyd93p7MEykrvX1D43uDACGD2W4at0m+vqHIhlf6fsDbNtzYHQHlpvmirUb2LbnQEnjk6DS9VTKMqJ4j2oJ852V8/0Wmmff84eLrp9i79HXP1T28Lcvmz/6I1pq3LUw/rOff+q8I9bTRNtbKdNv3D5Uk3VRy6SiUKpUsKuKma0ys/Vmtn5wcDDmsCRJ5i3oxsxiecxb0F1WTJW2x/6h4dENPWf4UIadQ8ORjK/0/QEG9haeZte+4ZLGJ0Gl66mUZUTxHpUqtT2G+c7K+X4LzXPghZGi66fYexRbp6UMNyPx7RKOXFeTxV3O9BmvzbqI85qKyWwHFuS9ng/sKDShu68B1gD09vZO2EdW6suO7U9z4XX3xbLsGy89o6z5Km2PXe3TaJ3SNGaDb53SxNz21kjGV/r+AJ1trQWnmTOztaTxSVDpeiplGVG8R6VKbY9hvrNyvt9C80xvbSm6fma2Tik4rtg6LXV40tslFF+/Ybe3iaZvttqsi1oeqbgFuDjoBfJqYEjXU0gj6Olq49oVi2mdkt38cueZe7raIxlf6fsDLJw9ndUrl46ZZvXKpSycPb2k8UlQ6XoqZRlRvEe1hPnOyvl+C80zs7W56Pop9h49XW1lD7/pF9u5/NxFiW6XcOS6+uEjzxyxniba3kqZ/hXz22uyLix7nWQMCzb7DnA2cBwwAHwCmALg7l82MyPbO2Q58BzwAXdfP9lye3t7ff36SSeTOmFmsR6pmKD9l3QlU7ntMddjYOfQMHPbW+npai/Yo6Dc8ZW+P7x4tfmufcPMmVn8avRi45Og0vVUyjKieI8SRNIew3xn5Xy/hebJ9f4otH6KvUclw+e2ZXt/DO5PbruEIz97rjdHqdtbKdPnen88d3CE7mh7fxRdSGxJRVyUVDSWek0qREJSe5QkKdoek9XnSURERFJLSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEoq6SinkLujGz2B7zFnTX+iOKiIgkVkutA4jSju1Pc+F198W2/BsvPSO2ZYuIiKRdXR2pEBERkdpRUiEiIiKRUFIhIiIikVBSISIiIpGINakws+Vm9piZPW5mVxYY325mPzSzR8ysz8w+EGc8IiIiEp/Ykgozawa+CLwJOAW4yMxOGTfZh4BfufsS4Gzgn81salwxiYiISHziPFJxOvC4u29194PAd4ELxk3jwEwzM2AG8DtgJMaYREREJCZxJhXzgKfzXm8PhuX7AnAysAN4FLjc3TPjF2Rmq8xsvZmtHxwcjCtekZKoPUqSqD1KksSZVFiBYT7u9X8DNgDHA0uBL5hZ2xEzua9x91537+3o6Ig6TpFQ1B4lSdQeJUniTCq2AwvyXs8ne0Qi3weA73vW48BvgZfHGJOIiIjEJM6k4iFgkZmdGFx8+W7glnHTPAWcC2BmncDLgK0xxiQiIiIxie3eH+4+YmYfBn4ENAPXu3ufmX0wGP9l4G+Br5vZo2RPl3zM3XfHFZOIiIjEJ9Ybirn7bcBt44Z9Oe/5DuC8OGMQERGR6lBFTREREYmEkgoRERGJhJIKERERiYSSChEREYmEkgoRERGJhJIKERERiYSSChEREYmEkgoRERGJhJIKERERiYSSChEREYmEkgoRERGJhJIKERERiYSSChEREYmEkgoRERGJhJIKERERiYSSChEREYmEkgoRERGJhJIKERERiYSSChEREYlErEmFmS03s8fM7HEzu7LINGeb2QYz6zOzH8cZj4iIiMSnJa4Fm1kz8EXgjcB24CEzu8Xdf5U3zTHAvwDL3f0pM5sTVzwiIiISrziPVJwOPO7uW939IPBd4IJx07wH+L67PwXg7rtijEdERERiFGdSMQ94Ou/19mBYvpcCs8zsXjP7hZldHGM8IiIiEqPYTn8AVmCYF3j/PwTOBaYB95vZz939N2MWZLYKWAXQ3d0dQ6gipVN7lCRRe5QkifNIxXZgQd7r+cCOAtPc7u4H3H038BNgyfgFufsad+91996Ojo7YAhYphdqjJInaoyRJnEnFQ8AiMzvRzKYC7wZuGTfNzcBZZtZiZkcDrwI2xxiTiIiIxCS20x/uPmJmHwZ+BDQD17t7n5l9MBj/ZXffbGa3AxuBDPBVd98UV0wiIiISnzivqcDdbwNuGzfsy+Ne/xPwT3HGISIiIvFTRU0RERGJhJIKERERiYSSChEREYlESUmFmV0y7nWzmX0inpBEREQkjUo9UnGumd1mZl1mthj4OTAzxrhEREQkZUrq/eHu7zGzC4FHgeeAi9z9v2KNTERERFKl1NMfi4DLgZuAbcD7gmJVIiIiIkDppz9+CFzt7pcCfwRsIVsxU0RERAQovfjV6e6+F8DdHfhnMxtfcltEREQaWKlHKqaZ2deCktqY2SnA6+ILS0RERNKm1KTi62Tv4dEVvP4N8NEY4hEREZGUKjWpOM7d15K96RfuPgIcji0qERERSZ1Sk4oDZjYbcAAzezUwFFtUIiIikjqlXqh5BXAL8BIz+y+gA3hnbFGJiIhI6kx4pMLMXmlmc939YbJdSf8GeAG4A9hehfhEREQkJSY7/XEdcDB4fgbwceCLwLPAmhjjEhERkZSZ7PRHs7v/Lnh+IbDG3W8CbjKzDbFGJiIiIqky2ZGKZjPLJR7nAnfnjSv1egwRERFpAJMlBt8Bfmxmu4HngZ8CmNkfoN4fIiIikmfCpMLd/87M7iJb9OqOoEQ3ZI9wfCTu4ERERCQ9Jq1T4e4/d/cfuPuBvGG/CXqETMjMlpvZY2b2uJldOcF0rzSzw2ambqoiIiIpVWrxq9DMrJlsT5E3AacAFwX3DCk03T+QLQMuIiIiKRVbUgGcDjzu7lvd/SDwXeCCAtN9BLgJ2BVjLCIiIhKzOJOKecDTea+3B8NGmdk84G3AlydakJmtMrP1ZrZ+cHAw8kBFwlB7lCRRe5QkiTOpsALDfNzrzwAfc/cJb07m7mvcvdfdezs6OqKKT6Qsao+SJGqPkiRx1prYDizIez0f2DFuml7gu2YGcBzwZjMbcfd1McYlIiIiMYgzqXgIWGRmJwLPAO8G3pM/gbufmHtuZl8HblVCISIikk6xJRXuPmJmHybbq6MZuN7d+8zsg8H4Ca+jEBERkXSJtdS2u98G3DZuWMFkwt3/NM5YREREJF5xXqgpIiIiDURJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQkmFiIiIREJJhYiIiEQi1qTCzJab2WNm9riZXVlg/J+Y2cbgcZ+ZLYkzHhEREYlPbEmFmTUDXwTeBJwCXGRmp4yb7LfAH7n7qcDfAmviikdERETiFeeRitOBx919q7sfBL4LXJA/gbvf5+7PBi9/DsyPMR4RERGJUZxJxTzg6bzX24NhxVwC/GehEWa2yszWm9n6wcHBCEMUCU/tUZJE7VGSJM6kwgoM84ITmr2ebFLxsULj3X2Nu/e6e29HR0eEIYqEp/YoSaL2KEnSEuOytwML8l7PB3aMn8jMTgW+CrzJ3ffEGI+IiIjEKM4jFQ8Bi8zsRDObCrwbuCV/AjPrBr4PvM/dfxNjLCIiIhKz2I5UuPuImX0Y+BHQDFzv7n1m9sFg/JeBq4HZwL+YGcCIu/fGFZOIiIjEJ87TH7j7bcBt44Z9Oe/5nwF/FmcMIiIiUh2qqCkiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRUFIhIiIikVBSISIiIpFQUiEiIiKRiDWpMLPlZvaYmT1uZlcWGG9m9rlg/EYzWxZnPCIiIhKflrgWbGbNwBeBNwLbgYfM7BZ3/1XeZG8CFgWPVwFfCv6KpNbISIa+/iH6h4bpap9GT1cbLS1j8/dMxtm25wADe4fpbGtl4ezpNDVZye8x2fwHDx5m444hdu4dpqutlVcc387Uqc2RfcZSTLYeSlkHk01T6Xqshlp/hmLLD/O+YWMs5zPl5hl6/iCHM7B7/wtFt59KP/v4aZ763QEG9r7AgYMjnHDsdE48LvznAxLfFovJ/zxd7a0czsCufeV9jtiSCuB04HF33wpgZt8FLgDyk4oLgBvc3YGfm9kxZtbl7v0xxiUSm5GRDOseeYar1m1i+FCG1ilNXLtiMSuWzBvdMWYyzu19O7li7YbRaVavXMrynrklbbyTzX/w4GHWbdzB1Te/GMM1FyxmxanHVy2xmGw9lLIOJpum0vVYDbX+DMWWf97JndyxeaCk9w0bYzmfKTfP9T97gncs6+ZTt/YV3X4q/ezj29jdjw2wZWA/n71rS9mf7wvvOY2DI57otlhM/ueZdfRULn7NCSWvi0LiPP0xD3g67/X2YFjYaURSo69/aPSHFGD4UIar1m2ir39odJptew6M7nxy01yxdgPb9hwo6T0mm3/jjqHRhCI3/uqbN7Fxx1DRZUZtsvVQyjqYbJpK12M11PozFFt+X/9Qye8bNsZyPlNunovPOGk0ocjNO377qfSzj29jG7cPjf6Ilvv5Nm4vfX0mTf7nefuy+aHWRSFxJhWF0hovYxrMbJWZrTez9YODg5EEJ1Kuidpj/9Dw6AaZM3wow86h4dHXA3sLT7Nr3zClmGz+nUXGD+wtbflRmGw9lLIOJpum0vVYDdX4DBO1x2LLL/b9FHrfsDGW85ly8zz/wsik20+pSm1jGafizxd2GUmS/3nMKv8ccSYV24EFea/nAzvKmAZ3X+Puve7e29HREXmgImFM1B672qfROmXsZtU6pYm57a2jrzvbWgtOM2dmK6WYbP6uIuM720pbfhQmWw+lrIPJpql0PVZDNT7DRO2x2PKLfT+F3jdsjOV8ptw8Rx/VMun2U6pS21izUfHnC7uMJBn/eSr9HHEmFQ8Bi8zsRDObCrwbuGXcNLcAFwe9QF4NDOl6Ckmznq42rl2xeHTDzJ0T7ulqH51m4ezprF65dMw0q1cuHb3YazKTzf+K49u55oKxMVxzwWJOPb696DKjNtl6KGUdTDZNpeuxGmr9GYotv6erreT3DRtjOZ8pN8837tvKJ87vmXD7qfSzj29jr5jfzuXnLqro871ifnvi22Ix+Z/npl9sD7UuCrHsNZLxMLM3A58BmoHr3f3vzOyDAO7+ZTMz4AvAcuA54APuvn6iZfb29vr69YUnMTMuvO6+CD/BWDdeegZxri85Upzf6STfZ0lXJRVqj7leDzuHhpnb3kpPV3vR3h+79g0zZ2b5vT+KzZ/r/ZG7Ev3UGvb+KLYeSlkHk01T6Xqshog+Q9ntsdjyw6y7sOu5nO8lN8/e5w8yMtr7o/D2U6pS21iu98dzB0foDtH7I3+5QOLbYjH5n2duW7b3x+D+8tpjrElFHJRUNJY0JhUiMVB7lCQp2h5VUVNEREQioaRCREREIpG60x9mNgg8WWT0ccDuKoYzEcVSWFpi2e3uyydbwCTtsdIYkkIxRqOSGKvVHsNK6npPYlz1FFPR9pi6pGIiZrbe3XtrHQcolmIUS/JimIxijEYaYgwrqZ8piXE1Skw6/SEiIiKRUFIhIiIikai3pGJNrQPIo1gKUyxjJSGGySjGaKQhxrCS+pmSGFdDxFRX11SIiIhI7dTbkQoRERGpESUVIiIiEonUJRXLly93srdH10OPOB8lUXvUo0qPkqg96lGlR1GpSyp2705a7RBpZGqPkiRqj1JrqUsqREREJJmUVIiIiEgkWmodQBRy94If2DtMZ1u67mMvIiKSFJX+nqY+qchknNv7dnLF2g0MH8rQOqWJ1SuXsrxnrhILERGREkXxe5r60x/b9hwYXQEAw4cyXLF2A9v2HKhxZCIiIukRxe9p6pOKgb3DoysgZ/hQhl37hmsUkYiISPpE8Xua+qSis62V1iljP0brlCbmzGytUUQiIiLpE8XvaeqTioWzp7N65dLRFZE7B7Rw9vQaRyYiIpIeUfyepv5CzaYmY3nPXF5+2Vns2jfMnJnq/SEiIhJWFL+nqU8qILsiTuqYwUkdM2odioiISGpV+nta1dMfZtZsZr80s1uD18ea2Z1mtiX4O6ua8YiIiEh0qn1NxeXA5rzXVwJ3ufsi4K7gdWiZjLN1cD/3P7GbrYP7yWQmvN+JiIiIBKL8Da3a6Q8zmw/8MfB3wBXB4AuAs4Pn3wDuBT4WZrkqfiUiIlKeqH9Dq3mk4jPA/wTyO8F2uns/QPB3TtiFqviViIhIeaL+Da1KUmFm5wO73P0XZc6/yszWm9n6wcHBMeNU/EqqbaL2KFJtao9Siah/Q6t1pOK1wFvNbBvwXeAcM/sWMGBmXQDB312FZnb3Ne7e6+69HR0dY8ap+JVU20TtUaTa1B6lElH/hlYlqXD3v3b3+e6+EHg3cLe7vxe4BXh/MNn7gZvDLlvFr0RERMoT9W9oretU/D2w1swuAZ4C3hV2ASp+JSIiUp6of0OrnlS4+71ke3ng7nuAcytdpopfiYiIlCfK39BaH6kQqbqRkQx9/UP0Dw3T1T6Nnq42WlpSfxscEZFRmYzz1O8OMLD3BQ4cHOGEY6dz4nHxH8VXUiENZWQkw7pHnuGqdZtG+2Rfu2IxK5bMU2IhInUhk3HufmyALQP7+exdW6paw0l7UWkoff1DowkFZLtOXbVuE339QzWOTEQkGtv2HGDj9qHRhAKqV8NJSYU0lP6hwn2ydw6promI1IeBvcNknJrUcFJSIQ2lq31awT7Zc9tV10RE6kNnWyvNRk1qOCmpkIbS09XGtSsWj+mTfe2KxfR0tdc4MhGRaCycPZ1XzG/n8nMXVb2Gky7UlIbS0tLEiiXzWDRnBjuHhpnb3kpPV7su0hSRutHUZJzzsk7+oGMGy7pn8dzBEbrV+0MkHi0tTSxZMIslC2odiYhIPJqajIXHzWDhcdWt36R/z0RERCQSOlIhDUfFr0QkbTIZZ9ueAwzsHaazrZXuWUfz1LPPjb5Oyu0plFRIQ1HxKxFJm0zGub1vJ1es3cDwoQwnzJ7GR85ZNGY/Vo3CVqXQXlQaiopfiUjabNtzYDShADj/1HlH7MeqUdiqFEoqpKGo+JWIpM3A3rH7LbPaFLYqhZIKaSgqfiUiadPZ1lpwvzX+ddyFrUqhpEIaiopfiUjaLJw9ndUrl47ut374yDNH7MeqUdiqFLpQUxqKil+JSNo0NRnLe+by8svOYte+YebMzPb+WNY9a/S1en+I1IiKX4lI2jQ1GSd1zOCkjheLWY1/nQT690xEREQioaRCREREIqGkQkRERCKhpEJEREQioaRCREREIqGkQkRERCKhpEJEREQioaRCREREIqHiV9JwRkYy9PUP0T80TFf7NHq62lRRU0SqKpNxtu05wMDeYTrbshUyn3r2udHXSamQGZaSCmkoIyMZ1j3yzOhtg3P3/lixZJ4SCxGpikzGub1v5+jtzE+YPY2PnLNozH5p9cqlLO+Zm7rEQntRaSh9/UOjGy5kbxd81bpN9PUP1TgyEWkU2/YcGE0oAM4/dd4R+6Ur1m5g254DtQyzLEoqpKH0Dw2Pbrg5w4cy7BwarlFEItJoBvaO3Q+ZUXC/tGtf+vZLSiqkoXS1Txu9XXBO65Qm5ra31igiEWk0nW2tBfdD41/PmZm+/ZKSCmkoPV1tXLti8egGnLumoqervcaRiUijWDh7OqtXLh3dD/3wkWeO2C+tXrmUhbOn1zLMsuhCTWkoLS1NrFgyj0VzZrBzaJi57a30dLXrIk0RqZqmJmN5z1xeftlZ7No3zJyZ2d4fy7pnjb5W7w+RlGhpaWLJglksWVDrSESkUTU1GSd1zOCkjhmjw8a/TqOq/HtmZq1m9qCZPWJmfWb2qWD4J83sGTPbEDzeXI14REREJHrVOlLxAnCOu+83synAz8zsP4Nx/9vdP13JwscXEUnrYSMREUkn/Q5lVSWpcHcH9gcvpwQPj2LZ44uIpLloiIiIpI9+h15UtavTzKzZzDYAu4A73f2BYNSHzWyjmV1vZrPCLnd8EZE0Fw0REZH00e/Qi6qWVLj7YXdfCswHTjezxcCXgJcAS4F+4J8LzWtmq8xsvZmtHxwcHDNufBERSG/REEmHidqjSLWpPdaefodeVPV+dO7+e+BeYLm7DwTJRgb4CnB6kXnWuHuvu/d2dHSMGVesiEgai4ZIOkzUHkWqTe2x9vQ79KJq9f7oMLNjgufTgDcAvzazrrzJ3gZsCrvs8UVE0lw0RERE0ke/Qy+qVu+PLuAbZtZMNpFZ6+63mtk3zWwp2Ys2twGXhl1woSIijXrVrYiIVJ9+h15Urd4fG4HTCgx/XxTLL1REREREpFr0O5RVFxU11T9YRESqRb85xaU+qVD/YBERqRb95kws9XdRUv9gERGpFv3mTCz1SYX6B4uISLXoN2diqU8q1D9YRESqRb85E0t9UqH+wSIiUi36zZlY6i/UVP9gERGpFv3mTCz1SQWof7CIiFSPfnOKS/3pDxEREUmG0EmFmXWa2dfM7D+D16eY2SXRh1a6TMbZOrif+5/YzdbB/WQyXstwREQkRfQbEp1yTn98HfhX4OPB698ANwJfiyimUFSIREREyqXfkGiVc/rjOHdfC2QA3H0EOBxpVCGoEImIiJRLvyHRKiepOGBms8neWRQzezUwFGlUIagQiYiIlEu/IdEq5/THFcAtwEvM7L+ADuCdkUYVQq4QSX6jUCESEREphX5DohX6SIW7Pwz8EXAGcCnQE9zavCZUiERERMql35BohT5SYWZvHzfopWY2BDzq7ruiCat0KkQiIiLl0m9ItMo5/XEJ8BrgnuD12cDPySYX17j7NyOKrWQqRCIiIuXSb0h0ykkqMsDJ7j4A2boVwJeAVwE/AaqeVIyMZOjrH6J/aJiu9mn0dLXR0qK6XiIijSyTcbbtOcDA3mE62wofgShlGildOUnFwlxCEdgFvNTdf2dmhyKKq2QjIxnWPfIMV63bNNrH+NoVi1mxZJ4SCxGRBlVK/QnVqIheOb+6PzWzW83s/Wb2fuBm4CdmNh34faTRlaCvf2g0oYBsV6Cr1m2ir79mvVxFqmLegm7MLJbHvAXdtf54IhUppf6EalREr5wjFR8C3g6cGbx+EOhy9wPA66MKrFT9Q4X7GO8cGmbJgmpHI1I9O7Y/zYXX3RfLsm+89IxYlitSLRPVn8hdO1HKNBJOOV1KHXgCOAS8DTgX2BxxXCXrap822hUop3VKE3Pb1cdYRKRR5epP5Btff6KUaSSckpMKM3upmV1tZpuBLwBPA+bur3f3L8QW4SR6utq4dsXiMX2Mr12xmJ6u9lqFJCIiNVZK/QnVqIhemNMfvwZ+CrzF3R8HMLO/iCWqEFpamlixZB6L5sxg59Awc9tb6elq10WaIiINrJT6E6pREb0wScU7gHcD95jZ7cB3gUSs+ZaWJpYsmKVrKEREZFQp9SdUoyJaJScV7v4D4AdBL48VwF8AnWb2JeAH7n5HPCGKREv90kXql7bv2grd+yPo5fFt4NtmdizwLuBKQEmFJJ76pYvUL23ftVfRhQfu/jt3v87dz4kqIJE4qV+6SP3S9l17uppRGspE/dJFJN20fdeekgppKNXslx5nxUszHcoVGU91J2qvnIqaIqmV65c+/pxrHP3S46x4Cap6KTJeNbdvKUxJhTQU9UsXqV/avmtPSYU0HPVLF6lf2r5rqyrXVJhZq5k9aGaPmFmfmX0qGH6smd1pZluCv7OqEY+IiIhEr1oXar4AnOPuS4ClwHIzezXZ+hZ3ufsi4K7gtUisMhln6+B+7n9iN1sH95PJeK1DEpEKaJtOjqqc/gjubLo/eDkleDhwAXB2MPwbwL3Ax6oRkzQmFccRqS/appOlal1KzazZzDYAu4A73f0BoNPd+wGCv3OqFY80JhXHEakv2qaTpWpJhbsfdvelwHzgdDNbXOq8ZrbKzNab2frBwcHYYpT6F0VxHLVHSZJGb48qeJUsVS9+5e6/J3uaYzkwYGZdAMHfXUXmWePuve7e29HRUa1QpQ5FURxH7VGSpNHbowpeJUu1en90mNkxwfNpwBuAXwO3AO8PJns/cHM14pHGlSuOk9sJqTiOSLppm06WatWp6AK+YWbNZBOZte5+q5ndD6w1s0uAp8je8VQkNiqOI1JftE0nS7V6f2wETiswfA9wbjViEMlRcRyR+qJtOjlUUVMazshIhr7+IfqHhulqn0ZPVxstLbq3nkhSZDLOtj0HGNg7TGfbi0ceig2X5FBSIQ1lZCTDukee4ap1m0b7tF+7YjErlsxTYiGSAMXqTpx3cid3bB5QPYqE015UGkpf/9BoQgHZrmdXrdtEX/9QjSMTEShed6Kvf0j1KFJASYU0lP6hwn3adw6pT7tIEhSrO1Fs21U9imRRUiENpat9WsE+7XPb1addJAmK1Z0otu2qHkWyKKmQhtLT1ca1KxaP6dN+7YrF9HS11zgyEYHidSd6utpUjyIFdKGmNJSWliZWLJnHojkz2Dk0zNz2Vnq62nWRpkhCTFR3QvUokk9JhTSclpYmliyYxZIFtY5ERAopVndC9SiSry6SioMHD7NxxxA79w7T1dbKK45vZ+rU5lqHJSIiJcrVoBh6/iCHM7B7/wuqI5NCqU8qDh48zLqNO7j65hfrDlxzwWJWnHq8EgsRkRTI1aa4/mdP8I5l3Xzq1j7VkUmp1H9LG3cMjSYUkO1idPXNm9i4Q3UHRETSIFeb4uIzThpNKEB1ZNIo9UnFziJ9mgf2qu+yiEga5GpTPP/CiOrIpFzqk4quIn2aO9vUd1lEJA1ytSmOPqpFdWRSLvVJxSuOb+eaC8bWHbjmgsWcerzqDoiIpEGuNsU37tvKJ87vUR2ZFEv9hZpTpzaz4tTjOem46aN3rjtVvT9ERFJjtAbF3Jnsff4g37rkVUHvD9WRSZvUJxWQTSx6Fx5b6zBE6kdTC2bxFBU6fv4Cnnn6qViWDTBvQTc7tj8dy7KbpxzF4UMvxLLsuNdL0uVqUEi61UVSISIRy4xw4XX3xbLoGy89I5bl5uzY/nSssad1vYhUQ10kFbmiKbnTHyrdKhMZGcnQ1z9E/9CwiuuIxCB/n9zV3srhDOzad+T+Wfvu+pP6pCJXNOWKtRtGi6WsXrmU5T1z1TjlCCMjGdY98gxXrduk4joiMcjfJ886eioXv+YEPnvXliP2z4D23XUo9XvRXNGU/GIpV6zdwLY9B2ocmSRRX//QaEIBKq4jErX8ffLbl80fTShg7P5Z++76lPqkYqBI8atd+1QsRY7UP1S4vai4jkg08vfJZhTdP2vfXZ9Sn1R0Fil+NWemiqXIkbrap6m4jkiMxu+Ti+2fte+uT6lPKnJFU/KLpaxeuZSFs6fXODJJop6uNq5dsVjFdURikr9PvukX27n83EUF98/ad9en1F+oOVo05bKz2LVvmDkzdQWxFNfS0sSKJfNYNGcGO4eGmaviOiKRGr9PntvWynmnzGVw/5H7Z+2760/qkwp4sWiKCqdIKVpamliyYBZLFtQ6kgYVY2EtSYZC++SXzDly/6x9d/2pi6RCJAzVqaixGAtrgYpI1VKhuhOAalE0ECUV0lBUp0IkHoVqBn3hPadxcMRVi6KBaC8qDUV1KkTiUajuxMbtQ6pF0WCUVEhDUZ0KkXgUqjuR8eJ1KqQ+KamQhqI6FSLxKFR3otmK16mQ+qSkQhqK6lSIxKNQ3YlXzG9XLYoGows1paGoToVIPIrVDAJUi6KBKKmQhqM6FSLxKFZ3QrUoGkdV/j0zswVmdo+ZbTazPjO7PBj+STN7xsw2BI83VyMeERERiV61jlSMAH/p7g+b2UzgF2Z2ZzDuf7v7pytZeKGCKzq8JsWo+JU0qjD7yvxpu9pbOZyBXfu0j5WJVSWpcPd+oD94vs/MNgPzolh2oYIrKq4ixaj4lTSqMPvK/GlnHT2Vi19zAp+9a4v2sTKpqu9FzWwhcBrwQDDow2a20cyuN7NZYZdXqOCKiqtIMSp+JY0qzL4yf9q3L5s/mlBMNp9IVZMKM5sB3AR81N33Al8CXgIsJXsk45+LzLfKzNab2frBwcEx4woVXFFxFSkmiuJXE7VHkWortT2G2VfmT2umAlZSuqolFWY2hWxC8W13/z6Auw+4+2F3zwBfAU4vNK+7r3H3Xnfv7ejoGDOuUMEVFVeRYqIofjVRexSptlLbY5h95fhptY+VUlWr94cBXwM2u/vqvOFdeZO9DdgUdtmFCq6ouIoUo+JX0qjC7Cvzp73pF9u5/NxF2sdKSarV++O1wPuAR81sQzDsb4CLzGwp4MA24NKwCy5WcEUXEEkhKn4ljSrMvnL8tHPbWjnvlLkM7tc+ViZWrd4fPwMKtcDbolh+sYIrIoWo+JU0qjD7ykLTvmSO9rEysbqoqKm6AyIihRWrTaH6PhKH1CcVqjsgIlJYsdoU553cyR2bB1TfRyKX+l9d1R0QESmsWG2Kvv4h1feRWKQ+qYii7oCISD0qVpui2H5TtSekUqlPKqKoOyAiUo+K1aYott9U7QmpVOqTCtUdEBEprFhtip6uNtX3kVik/kJN1R0QESlsotoUqu8jcUh9UgGqOyAiUkyx2hSq7yNx0L/zIiIiEgklFSIiIhIJc/daxxCKmQ0CTxYZfRywu4rhTESxFJaWWHa7+/LJFjBJe6w0hqRQjNGoJMZqtcewkrrekxhXPcVUtD2mLqmYiJmtd/feWscBiqUYxZK8GCajGKORhhjDSupnSmJcjRKTTn+IiIhIJJRUiIiISCTqLalYU+sA8iiWwhTLWEmIYTKKMRppiDGspH6mJMbVEDHV1TUVIiIiUjv1dqRCREREakRJhYiIiERCSYWIiIhEInVJxfLlyx3QQ4+4HyVRe9SjSo+SqD3qUaVHUalLKnbvTlpBMmlkao+SJGqPUmupSypEREQkmZRUiIiISCRaah1AFDIZZ9ueAwzsHaazrZWFs6fT1GS1DktEZFLaf0k9SX1Skck4t/ft5Iq1Gxg+lKF1ShOrVy5lec9cbZgikmjaf0m9Sf3pj217DoxukADDhzJcsXYD2/YcqHFkIiIT0/5L6k3qk4qBvcOjG2TO8KEMu/YN1ygiEZEXZTLO1sH93P/EbrYO7ieTebFHnvZfUm9Sf/pjzsxWWqc0jdkwW6c00TGjtYZRiYhMfnqjs63w/mvOTO2/JJ1Sf6SiuQkuP3cRrVOyH6V1ShOXn7uI5tR/MhFJu8lObyycPZ3VK5eO2X+tXrmUhbOn1yxmkUqk/khF/9AwN9z/JJeceRJm4A433P8kp3Ufw8LjZtQ6PBFpYBOd3jipYwZNTcbynrm8/LKz2LVvmDkz1ftD0q1qSYWZXQ+cD+xy98XBsGOBG4GFwDZgpbs/G2a5nW2tPPvcQb54z+Ojw3T4UESSoJTTG01NxkkdMzipQ/8ESfpV8yTB14Hl44ZdCdzl7ouAu4LXoejwoYgklfZP0miqdqTC3X9iZgvHDb4AODt4/g3gXuBjYZarw4ciklTaP0mjqfU1FZ3u3g/g7v1mNqfQRGa2ClgF0N3dfcR4HT6UapqsPYrki3v/pPYoSZKKPhLuvsbde929t6Ojo9bhSINTe5QkUXuUJKl1UjFgZl0Awd9dNY5HREREylTrpOIW4P3B8/cDN9cwFhEREalANbuUfofsRZnHmdl24BPA3wNrzewS4CngXdWKRxqX7gopUVJ7EnlRNXt/XFRk1LnVikFEd4WUKKk9iYxV69MfIlX1292Fyyb/drfuCinhqT2JjKWkQhrKk787ULBs8lO/04+AhKf2JDKWkgppKNOntoxWN8xpndLE0VNrXbJF0kjtSWQsJRXSUDrbjip4V9vOtqNqHJmkkdqTyFhKp6WhdB87nUWdM1j1upPIODQZLOqcQfexuheDhKf2JDKWkgppKE1Nxjkv6+Sk42boXgxSMbUnkbGUVEjD0b1iJEpqTyIvUlIhDUfFiiQMtReR0impkIaiYkUShtqLSDjq/SENZduewsWKtu1RXQE5ktqLSDhKKqShDOwdLlisaNe+4RpFJEmm9iISjpIKaSidba0FixXNmdlao4gkydReRMJRUiENpXvW0Vy7YvGYYkXXrlhM96yjaxyZJJHai0g4ulBTGspTzz7H5+/ewiVnnoQZuMPn797Csu5Z6hIoR1B7EQlHSYU0lIG9wzy553m+eM/jY4bv2jesHwk5gtqLSDg6/SENRefIJQy1F5FwlFRIQ1k4ezqrVy4dc4589cqlLJytezXIkdReRMLR6Q9pKE1NxvKeubz8srN0rwaZlNqLSDhKKqTh6F4NEobai0jpQicVZnYCsMjd/4+ZTQNa3H1f9KGJxEP3cmgc+q5FqitUUmFmfw6sAo4FXgLMB74MnBt9aCLR070cGoe+a5HqC3uh5oeA1wJ7Adx9CzCnkgDM7C/MrM/MNpnZd8xMl1VLbHQvh8ah71qk+sImFS+4+8HcCzNrAbzcNzezecBlQK+7LwaagXeXuzyRyeheDo1D37VI9YVNKn5sZn8DTDOzNwLfA35YYQwtwfJagKOBHRUuT6Qo1R1oHPquRaovbFJxJTAIPApcCtwGXFXum7v7M8CngaeAfmDI3e8od3kik9G9HBqHvmuR6gvb+2MacL27fwXAzJqDYc+V8+ZmNgu4ADgR+D3wPTN7r7t/a9x0q8heIEp3d3c5byUCRHMvB7XHdGiU+3aoPUqShE0q7gLeAOwPXk8D7gDOKPP93wD81t0HAczs+8GyxiQV7r4GWAPQ29tb9jUcIlHcy0HtMR0a5b4dao+SJGFPf7S6ey6hIHheybHEp4BXm9nRZmZku6ZurmB5IhPSefbGoe9apPrCJhUHzGxZ7oWZ/SHwfLlv7u4PAP8OPEz2Oo0mgoxbJA66l0Pj0HctUn1hT398lOx1D7keGl3AhZUE4O6fAD5RyTJESqV7OTQOfdci1RcqqXD3h8zs5cDLAAN+7e6HYolMJCa6l0Pj0HctUl3l3FDslcDCYN7TzAx3vyHSqERERCR1wt7745tk7/mxATgcDHZASYWIiEiDC3ukohc4xd3VbUlERETGCNv7YxMwN45AREREJN3CHqk4DviVmT0IvJAb6O5vjTQqERERSZ2wScUn4wiiUpmMs23PAQb2DtPZpm5jIo1C275IsoTtUvpjMzsBWOTu/8fMjiZ7u/KayWSc2/t2csXaDQwfyowWuFneM1c7F5E6pm1fJHlCXVNhZn9OtgLmdcGgecC6iGMKZdueA6M7FYDhQxmuWLuBbXsO1DIsEYmZtn2R5Al7oeaHgNcCewHcfQswJ+qgwhjYOzy6U8kZPpRh177hGkUkItWgbV8kecImFS+4+8HcCzNrIVunomZ00yCR+pXJOFsH93P/E7vZOrifTObF3Y22fZHkCZtU/NjM/gaYZmZvBL4H/DD6sErXPetorl2xeMxNg65dsZjuWZXcPFVEai13zcSbP/dTLvrKA7z5cz/l9r6do4mFbhgmkjxhe39cCVxC9o6ilwK3AV+NOqgwnnr2OT5/9xYuOfMkzMAdPn/3FpZ1z1K9f5EUK3bNxMsvO4uTOmbohmEiCRS290cG+ErwSISBvcM8ued5vnjP42OG79o3rKRCJMUmumYit23rhmEiyVJSUmFma919pZk9SoFrKNz91MgjK1HuvGr+zkfnVUXST9u2SPqUek3F5cHf84G3FHjUjM6ritQnbdsi6VPSkQp37zezZuBr7v6GmGMKRedVReqTtm2R9Cn5mgp3P2xmz5lZu7sPxRlUWDqvKlKftG2LpEvY3h/DwKNmdicwWrbO3S+LNCoRERFJnbBJxX8EDxEREZExwnYp/YaZTQO63f2xmGISERGRFAp7Q7G3ABuA24PXS83slhjiEhERkZQJW6b7k8DpwO8B3H0DcGKlQZjZMWb272b2azPbbGavCTP/RPcHEJFk0nYrUn/CXlMx4u5DZmO6dEWxJ/gscLu7v9PMpgIl37gjd3+AXDnfXF/25T1z1fVMJKG03dafeQu62bH96ViWffz8BTzz9FOxLFuiFTap2GRm7wGazWwRcBlwXyUBmFkb8DrgTwGCu6AenGiefJPdH0BEaiOTcbbtOcDA3mE628bWmNB2W392bH+aC6+r6OegqBsvPSOW5Ur0wp7++AjQA7wAfAfYC3y0whhOAgaBfzWzX5rZV81sTMk8M1tlZuvNbP3g4OCYmYvdH2Bg73CFYYkUNlF7lKzJ7jA60X09JBy1R0mSUEmFuz/n7h8HzgVe7+4fd/dK9wItwDLgS+5+Gtn6F1eOe9817t7r7r0dHR1jZj56astoGd+c1ilNHD21ucKwRAqbqD1KVrEjEdv2ZMvb5O7rkU/39SiP2qMkSdjeH68Mbiq2kWwRrEfM7A8rjGE7sN3dHwhe/zvZJKMkBw8f5rJzFo25P8Bl5yzi0OHMJHOKSFwmOxKh+3qI1Kew11R8Dfh/3f2nAGZ2JvCvQNl3KXX3nWb2tJm9LKh9cS7wq1Lnnz39KG5c/xSXnHkSZuAON65/iuWL55YbkohUaLI7jOq+HiL1KWxSsS+XUAC4+8/MbF8EcXwE+HbQ82Mr8IFSZ1w4ezofW37yEVeR6z8ekdrJHYmYaLvUfT1E6k/YpOJBM7uO7EWaDlwI3GtmywDc/eFyggjqXfSWM6/+4xFJHm2XIo0pbFKxNPj7iXHDzyCbZJxTaUDl0H88Ismj7VKk8YS998fr4wpERERE0i1s749vmll73usTzOyu6MMKR+V+RZJH26VI4wl7+uNnwANmdgUwD/gfwF9GHlUIKvcrkjzaLkUaU9jiV9cBfwbcDFwDvM7dfxhHYKWarMiOiFSftkuRxhT29Mf7gOuBi4GvA7eZ2ZIY4iqZyv1KWDosH42J1qO2S5HGFPb0xzuAM919F/AdM/sB8A1e7BVSdZMV2RHJp8Py0ZhsPWq7FGlMYU9/rAgSitzrB4HTI48qBJX7lTB0WD4ak61HbZcijamkIxVmttbdVwbP/8HdP5Y3+lbgvDiCK4WK7EgYEx2WVz2FsSa6dflk61HbpUhjKvX0x6K8528E8pOKmt8WT0V2pFRzZhY+LN8xQ4fl80VxekPbpUjjKfX0x0RXsukqN0mN5ia4/Nyxd7W9/NxFNIc6EVj/dHpDRMpR6pGKo83sNLJJyLTguQWPaXEFJxK1/qFhbrj/yTF3tb3h/ic5rfsYFh6n/6hzdHpDRMpRalLRD6wOnu/Me557LZIKnW2tPPvcQb54z+Ojw9Qr4Ug6vSEi5SjpoK+7v36iR9xBikRFh+1Lo/UkIuUIVafCzN4F3O7u+8zsKmAZ8Lfu/stYohOJWFOTcd7Jndy46tX0Dw3T1d5KT1d7wx22n6hnB6hXlYiUJ2zxq//P3b9nZmcC/w34NPBl4FWRRyYSg0zGuWPzQEMXvyq1AJhOb4hIWGGveT8c/P1j4EvufjMwNdqQROKj4ldaByISn7BJxTNmdh2wkux9P44qYxkiNaN7UmgdiEh8wiYEK4EfAcvd/ffAsWRvfy6SCrleDfkarfeH1oGIxKXkpMLMmoAH3f377r4FwN373f2O2KITiZh6NWgdiEh8Sr5Q090zZvaImXW7+1NxBiUSF/X+0DoQkfiE7f3RBfSZ2YPA6FVd7v7WSoIws2ZgPfCMu59fybJEJqLeH1oHIhKfsEnFp2KJAi4HNgNt5cw8MpKhr38o+K9rGj1dbbS06PpROVKxng8vv+ysuuo6OdE20SjrQESqL1RS4e4/jjoAM5tPtovq3wFXhJ1/ZCTDukee4ap1m0b/67p2xWJWLJmnxEKO0Ai3Pp9sm2iEdSAitRHqV9fMXm1mD5nZfjM7aGaHzWxvhTF8BvifQGaS6Qrq2zE0uvOE7M7xqnWb6NsxVGFYUo8aoefDZNtEI6wDEamNsP/KfwG4CNhC9u6kfxYMK4uZnQ/scvdfTDLdKjNbb2brBwcHx4x7Zuj5gv91PTP0fLlhSR2LoufDRO0xCSbbJtT7o74kvT1KYwl7TQXu/riZNbv7YeBfzey+Ct7/tcBbzezNQCvQZmbfcvf3jnvPNcAagN7eXs8fd9yMowreTfG46UdVEJbUqyh6PkzUHqtlont3TLZN6L4e9SUJ7VEkJ2xS8ZyZTQU2mNk/kr0letn/3rj7XwN/DWBmZwN/NT6hmExn21F84i09fOqHfaPnjz/xlh4625VUyJHqoefDZPfuKGWb0H09RCQOYZOK95E9ZfJh4C+ABcA7og4qjAWzpnP8MQf49DuXcODgCNOntjBzWjMLZulQrhypHno+TPYZtE2ISK2E7f3xpJlNA7rcPdLupe5+L3Bv2Pmamoyz/mAO2/Yc0KFcmVQ99HyY7DNom5BC5i3oZsf2p2sdhkQozu/0+PkLeObp8HUuQyUVZvYWsrc7nwqcaGZLgWsqLX5VKR3KlVLlej6Mv94gTT0fSvkM2iZkvB3bn+bC6yq5BG5iN156RmzLlsLi/E7L/T7D9v74JHA68HsAd98ALCzrnUVqoB56PtTDZxCR+hT2mooRdx8y02FUSad66PlQD59BROpTSUmFmd0GfAjYZGbvAZrNbBFwGRDf8TSRGNTDqYF6+AwiUn9KPf3xdeBHwDZgMfAC8G/AENn7doikxshIhkeefpbbN/XzyNO/Z2SkrGKuscpknK2D+7n/id1sHdxPJuOhxouI1EJJRyrcfa2Z/QdwNbAc+CaQ24t9CFgdT3gi0UrDvWImq0Mx2XgRkVoJsxc9RPZ250cBM/IeM2OISyQWff1F7ovRn5x7xRSrQ7Ftz4GSxouI1Eqp11QsJ3s04hZgmbs/F2tUIU1UslgkX/9Q4RoPO4eGWbKgRkGNM1kdinqotSEi9anU3h8fB97l7n1xBlMOHQqWMLrapxWs8TC3PTl1KiarQzFnZuHxHTOS8xlEItXUQly9DpunHMXhQy/Esmwov4hUWpV6TcVZcQdSrnoouyzVc3LnTK65YDFX3/ziNRXXXLCYkzvbah3aqO5ZR3PtisVHXPfRPetoAJqb4PJzF/HZu7aMjr/83EU0J+OSEJHoZUZiLfKkomDRCX2X0qTRoWAJY/vQ86x96En+8Z1LeP7gCNOmtnDDfVvpPWFWYtrLU88+x+fv3sIlZ56EGbjD5+/ewrLubIz9Q8PccP+TY8bfcP+TnNZ9DAuPS8ZnEJHGlPqkoh7KLkv1DOwdZv2TQ6x/8pdjhicpCR3YO8yTe57ni/c8PmZ4LsbOtlaefe7gmPFq8yKSBKk/YKqSxRJGLgnNl7Qf5MliVJsXkaRK/ZEKlSyWMHI/yOMv7E3SD/JkMarNi0hSpT6pAJUsltKl4Qe5lBjV5kUkieoiqRAJIw0/yGmIUURkvNRfUyEiIiLJoKRCREREIqHTHyIiInGJsRpoEtVFUqF7f0jaqM2KNIiYq4EmTeqTCt37Q9JGbVZE6lXqr6nQbaAlbdRmRaRe1TypMLMFZnaPmW02sz4zuzzM/BPd+0OkkEzG2Tq4n/uf2M3Wwf1kMl7V91ebFZF6lYTTHyPAX7r7w2Y2E/iFmd3p7r8qZWbd+0PCSMKpB926XETqVc2PVLh7v7s/HDzfB2wG5pU6v+6DIGEk4dRD7tbl+W1Wty4XkXqQhCMVo8xsIXAa8MC44auAVQDd3d1j5klD2WVJjolOPZRavXKi9lgK3bpcolRpexSJUmKSCjObAdwEfNTd9+aPc/c1wBqA3t7eI06Aq6SxlCqK02WTtcdSYtCtyyUqlbZHkSgl4oCrmU0hm1B8292/X+t4pH4l4XRZEmIQEYlDzY9UWLbU2NeAze6+utbxSH1LwumyJMQgIhKHmicVwGuB9wGPmtmGYNjfuPtttQtJ6lkSTpclIQYRkajVPKlw958BFf2LppLHkiRqjyLSqGqeVFQqCXUHRHLUHkWkkSXiQs1KJKHugEiO2qOINLLUJxUqeSxhxVmmu9T2WOtS4SIicUj96Q+V6ZYw4j49UUp71CkSEalXqT9SoT7/EkbcpydKaY86RSIi9Sr1RyqamozzTu7kxlWvpn9omK72Vnq62vUfnxQURZnuiZTSHuOOQUSkVlKfVGQyzh2bB3QoWUoS9+myUtqjTtmJSL1K/ekPHUqWMOI+XVZKe9QpOxGpV6k/UqFDyRJG3CWyS2mPKtMtIvUq9UmFDiVLWHGWyC61PapMt4jUo9Sf/tChZEkStUcRaWSpP1KhQ8mSJGqPItLIUp9UgA4lS7KoPYpIo0r96Q8RERFJBiUVIiIiEglzT9eNjMxsEHiyyOjjgN1VDGciiqWwtMSy292XT7aASdpjpTEkhWKMRiUxVqs9hpXU9Z7EuOoppqLtMXVJxUTMbL2799Y6DlAsxSiW5MUwGcUYjTTEGFZSP1MS42qUmHT6Q0RERCKhpEJEREQiUW9JxZpaB5BHsRSmWMZKQgyTUYzRSEOMYSX1MyUxroaIqa6uqRAREZHaqbcjFSIiIlIjqUkqzGy5mT1mZo+b2ZUFxpuZfS4Yv9HMlpU6bwyx/EkQw0Yzu8/MluSN22Zmj5rZBjNbX4VYzjazoeD9NpjZ1aXOG0Ms/yMvjk1mdtjMjg3GRbZezOx6M9tlZpuKjK9aW8lb7gIzu8fMNptZn5ldHgz/pJk9k7de3hzVe5YRY6uZPWhmjwQxfioYfqyZ3WlmW4K/sxIYY2LWY16szWb2SzO7NXidmPUYRrG2O26aovuZGOOacJ8x0XYeY0wvy1sHG8xsr5l9dNw0sa+rQvvAUttfxftAd0/8A2gGngBOAqYCjwCnjJvmzcB/Aga8Gnig1HljiOUMYFbw/E25WILX24DjqrhezgZuLWfeqGMZN/1bgLtjWi+vA5YBm4qMr0pbGfeeXcCy4PlM4DfAKcAngb+K4j0iiNGAGcHzKcADwfr5R+DKYPiVwD8kMMbErMe8WK8A/i23/SVpPYb8HAXb7rhpCu5nYo5rwn1Gse28ivE1AzuBE6q9rgrtA0tpf1HsA9NypOJ04HF33+ruB4HvAheMm+YC4AbP+jlwjJl1lThvpLG4+33u/mzw8ufA/Arer6JYYpo3iuVdBHyngvcryt1/Avxugkmq1VbyY+p394eD5/uAzcC8KJYdlWB97A9eTgkeTnYdfCMY/g1gRfWjy5ogxkQxs/nAHwNfzRucmPUYRhrabhHFtvNqORd4wt2rWYwMKLoPLKX9VbwPTEtSMQ94Ou/1do5s1MWmKWXeqGPJdwnZbDnHgTvM7BdmtqqCOMLE8prgcPF/mllPyHmjjgUzOxpYDtyUNzjK9TKZarWVgsxsIXAa2f+yAT4cHJ69vtaHxIND9huAXcCd7v4A0Onu/ZD9gQHm1DDEYjFCgtYj8BngfwKZvGGJWo/lKNB28xXaz8Rpsn1GVbbnCbyb4v84VXtdQWntr+J1lpakotB9o8f/d1JsmlLmjTqW7IRmryebVHwsb/Br3X0Z2dMiHzKz18Ucy8NkD78tAT4PrAsxb9Sx5LwF+C93z8+ko1wvk6lWWznyjc1mkE2mPurue4EvAS8BlgL9wD9H+X5hufthd19K9uja6Wa2uJbxFFIkxsSsRzM7H9jl7r+oVQxxKNB28xXbz8Rpsn1G7NtzMWY2FXgr8L0Co2uxrkpV8TpLS1KxHViQ93o+sKPEaUqZN+pYMLNTyR76vMDd9+SGu/uO4O8u4AdkDzfFFou7780dLnb324ApZnZcqZ8jyljyHJHBR7xeJlOttjKGmU0hu1P+trt/H8DdB4IfyQzwFeL93CVz998D95I9ojSQO2wc/N1Vu8helB9jwtbja4G3mtk2soePzzGzb5HQ9ViKQm033wT7mdiUsM+IdXuexJuAh919YPyIWqyrQCntr+J1lpak4iFgkZmdGGSA7wZuGTfNLcDFwRW/rwaGgkM8pcwbaSxm1g18H3ifu/8mb/h0M5uZew6cBxTsoRBhLHPNzILnp5P9zveUMm/UsQQxtAN/BNycNyzq9TKZarWVUcF38DVgs7uvzhuef473bcT7uSdkZh1mdkzwfBrwBuDXZNfB+4PJ3k/ed1dtxWJM0np097929/nuvpBsG7rb3d9LgtZjGMXa7rhpiu1n4oqplH1Gse28GopeM1btdZWnlPZX+T4wzFWdtXyQvZL3N2SvTP14MOyDwAeD5wZ8MRj/KNA70bwxx/JV4FlgQ/BYHww/iezVtI8AfVWK5cPBez1C9qLRM2q1XoLXfwp8d9x8ka4XshtzP3CIbOZ9Sa3aSt5yzyR7GHFjXrt4M/DNIIaNZDfermpuV+NiPBX4ZRDLJuDqYPhs4C5gS/D32ATGmJj1OC7es3mx90di1mPIz1Cs7Za0n4kppoL7jFK385hjO5psktCeN6yq66rIPrBg+wOOB27Lm7eifaAqaoqIiEgk0nL6Q0RERBJOSYWIiIhEQkmFiIiIREJJhYiIiERCSYWIiIhEQklFnTCzj1v2DoIbLXvnu1fVOiZpXGb2NjNzM3t5rWOR2glqRPzMzN6UN2ylmd0ew3vda9m7a+bu/vnvUb/HuPc7Pu73SCN1Ka0DZvYaYDVwtru/EFRnm+pBxTmRajOztWTvbnmXu3+yxuFIDQVl1L9H9p4hzWTrXCx39yfKWFazux8uMu5esneqPeI26FEzsxZ3H4n7fdJIRyrqQxew291fAHD33e6+w8z+0Mx+bNkb7vzIzLrMrD3I5l8GYGbfMbM/r2n0Ulcse4+I15ItuPPuYFiTmf1LcDTtVjO7zczeGYw7op3WMHyJmLtvAn5I9h5InwC+BXzczB4ys1+a2QWQvVmZmf3UzB4OHmcEw882s3vM7N+AR4Nqmv9h2RtybTKzCyd6fzO72cwuDp5fambfDp7fa2afMbP7guWcHgyfbtmb0o2P70/N7Htm9kOyNzJbaGabgnHNZvZPwTwbzezSvNjvNbN/N7Nfm9m386ppvjJ470fM7EEzm1lsOalSjQpjesT7AGaQzf5/A/wL2TLYU4D7gI5gmguB64PnbwTuJ7vDv73W8etRXw/gvcDXguf3AcuAdwK3kf1HZi7ZirPvnKid6lE/D2A68BjZypb/C3hvMPyYYL81nWwlytZg+CJerER8NnAAODF4/Q7gK3nLbg/+3hu8x4bg8U/B8E7gceCs4L2OzZv+K8Hz1wGbguf/f5H4/pRsdcrc/Avz5lkFXBU8PwpYD5wYxD5E9h4aTcF+90xgKrAVeGUwTxvQUmw5tf7+wjxakNRz9/1m9odkN5rXAzcC1wKLgTuDxLiZbNlW3P1OM3sX2RK2S2oStNSzi8je+huyN9S6iGzy8D3P3vBrp5ndE4x/GUXaqdQPdz9gZjcC+4GVwFvM7K+C0a1AN9kbV33BzJYCh4GX5i3iQXf/bfD8UeDTZvYPZEug/zRvuj/xcac/3H3AzK4G7gHe5mPvjvydYJqfmFmbZe8rcx7ZG8KNjw/gznHz55wHnJo7+ga0k02MDgaxbwcwsw1kk5EhoN/dHwref28wvthycp898ZRU1AnPnme8F7jXzB4FPgT0uftrxk9rZk3AycDzwLFks2+RipnZbOAcYLGZOdkkwcneRbLgLBRpp1J3MsHDgHe4+2P5I83sk8AA2X90moDhvNEHck/c/TfBP1FvBv6Xmd3h7tdM8t6vIHs/juPHDR9/UaFPEN+r8uMYx4CPuPuPxs1zNvBC3qDDZH93rcB7F11OmuiaijpgZi8zs0V5g5YCm4GO4CJOzGyKmfUE4/8iGH8RcL1lb2ssEoV3Aje4+wnuvtDdF5D9L2s38I7g2opOsoeFIXu4ulg7lfr0I+AjedcWnBYMbyf733sGeB/ZhPQIZnY88Jy7fwv4NNnTa0UF10q8ieyFon9lZifmjb4wmOZMsncxHZogvsk+03/P7UvN7KWWvXtqMb8GjjezVwbTzzSzljKWkzg6UlEfZgCfDw7djZA9f7gKWAN8zrK3G28BPmNmh4A/A053931m9hPgKrIXUIlU6iLg78cNu4nskbHtZO8s+hvgAbI78YPBod4x7ZTsXRylPv0t2e94Y/DDvQ04n+z1YDcFp2bvofhRgVcA/2RmGbJ34fzveeO+bWbPB893A38MfAX4gGcvXv9Lsv9InRNM86yZ3Uf2mob/Z5L4JvJVsqc1Hg7mGQRWFJs4aPcXkt1vTyN71PgNYZeTROpSKiJVYWYzgut/ZgMPAq919521jksak1WxC2oj0ZEKEamWW4OjaVOBv1VCIVJ/dKRCREREIqELNUVERCQSSipEREQkEkoqREREJBJKKkRERCQSSipEREQkEkoqREREJBL/FxlpjaeNXTagAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 540x540 with 12 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.pairplot(X)#绘制配对图"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2da8b788",
   "metadata": {},
   "source": [
    "由此可见，Age与YearsExperience具有极强的关联性。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad29db5d",
   "metadata": {},
   "source": [
    "#### （3）使用VIF（必须使用）"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d4b0f9a",
   "metadata": {},
   "source": [
    "安装VIF库指令：\n",
    "\n",
    "pip install statsmodels\n",
    "\n",
    "or\n",
    "\n",
    "conda install -c conda-forge statsmodels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "2a74e604",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1349098210011803\n",
      "12.138592602889707\n",
      "10.367631468765076\n"
     ]
    }
   ],
   "source": [
    "#逐一查看各特征与其他特征的VIF(方差扩大因子)\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    "\n",
    "print(variance_inflation_factor(X.values, 0))\n",
    "print(variance_inflation_factor(X.values, 1))\n",
    "print(variance_inflation_factor(X.values, 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "3ae53e3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#还有一种更方便的方法是用封装好的VIF接口：\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    " \n",
    "def vif(X):\n",
    " \n",
    "    # Calculating VIF\n",
    "    vif = pd.DataFrame()\n",
    "    vif[\"variables\"] = X.columns\n",
    "    vif[\"VIF\"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]\n",
    " \n",
    "    return(vif)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "1ea889c3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sex</td>\n",
       "      <td>2.134910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Age</td>\n",
       "      <td>12.138593</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>YearsExperience</td>\n",
       "      <td>10.367631</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         variables        VIF\n",
       "0              Sex   2.134910\n",
       "1              Age  12.138593\n",
       "2  YearsExperience  10.367631"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 将数据样本X（全部的特征变量列）直接作为参数传入，就可以得到各特征相对于其他特征的VIF值。\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34f76a56",
   "metadata": {},
   "source": [
    "据图分析，Age和YearsExperience都大于10，关联性极强。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "28068c68",
   "metadata": {},
   "outputs": [],
   "source": [
    "#在X中除去Age列\n",
    "X=X.drop('Age',axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2558be01",
   "metadata": {},
   "source": [
    "drop函数是一个在数据处理中常用的函数，特别是在处理Pandas库中的DataFrame数据类型时。它的主要功能是删除数据集中的多余数据。\n",
    "\n",
    "它的语法格式为：DataFrame.drop (labels=None, axis=0, index=None, columns=None, level=None, inplace=False, errors='raise')。\n",
    "\n",
    "其中，一些主要的参数含义如下：\n",
    "- labels: 表示待删除的行名或列名；\n",
    "- axis: 表示删除时所参考的轴，0代表行，1代表列；\n",
    "- index: 表示待删除的行名；\n",
    "- columns: 表示待删除的列名；\n",
    "- level: 用于多级列表时，暂不详细说明；\n",
    "- inplace: 这是一个布尔值，默认为False，意味着返回的是数据的副本。如果设为True，则返回删除相应数据后的版本；\n",
    "- errors: 这个参数一般用不到，这里不作解释。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "42db7e01",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sex</td>\n",
       "      <td>1.804751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>YearsExperience</td>\n",
       "      <td>1.804751</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         variables       VIF\n",
       "0              Sex  1.804751\n",
       "1  YearsExperience  1.804751"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#再次检测VIF\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "450bb050",
   "metadata": {},
   "source": [
    "剩余变量的VIF都小于4，正常。\n",
    "\n",
    "注意：每次删除一个变量都需要再次重新计算VIF，因为VIF是随特征变化而变化的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26350efe",
   "metadata": {},
   "source": [
    "#### （4）最小二乘法(又称最小平方法)\n",
    "\n",
    "函数名：\n",
    "statsmodels.regression.linear_model.OLS Ordinary Least Squares\n",
    "\n",
    "最小二乘法，又称最小平方法，是一种数学优化技术。它通过最小化误差（真实目标对象与拟合目标对象的差）的平方和寻找数据的最佳函数匹配。这种方法可以简便地求得未知的数据，并使得这些求得的数据与实际数据之间误差的平方和为最小。\n",
    "\n",
    "在线性回归中，最小二乘法主要用来求解回归系数，使得所有样本点到回归直线的距离的平方和最小。对于非线性回归问题，可以通过将非线性模型转化为线性模型来使用最小二乘法。此外，最小二乘法还可以应用于测量数据处理的平差公式中，如矩阵的最小二乘法常用于此。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "f7465f0b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "        if (window._pyforest_update_imports_cell) { window._pyforest_update_imports_cell('import statsmodels.api as sm'); }\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X2=sm.add_constant(X1)#在X1的第1列添加常数1。例如多项式回归中：1+ax+bx^2+cx^3+……"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "4b9d424e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "        if (window._pyforest_update_imports_cell) { window._pyforest_update_imports_cell('import statsmodels.api as sm'); }\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#用响应列y和特征列X2通过最小二乘法模型做拟合 \n",
    "model=sm.OLS(y,X2).fit() #Ordinary Least Squares最小二乘法(又称最小平方法)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2d955ce7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>Salary</td>      <th>  R-squared:         </th> <td>   0.966</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.962</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   245.4</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Sat, 20 Jan 2024</td> <th>  Prob (F-statistic):</th> <td>3.49e-19</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>16:33:15</td>     <th>  Log-Likelihood:    </th> <td> -297.95</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>    30</td>      <th>  AIC:               </th> <td>   603.9</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    26</td>      <th>  BIC:               </th> <td>   609.5</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "         <td></td>            <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th>           <td>-6.833e+04</td> <td> 3.72e+04</td> <td>   -1.838</td> <td> 0.078</td> <td>-1.45e+05</td> <td> 8094.672</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Sex</th>             <td> -311.4461</td> <td> 1984.473</td> <td>   -0.157</td> <td> 0.877</td> <td>-4390.590</td> <td> 3767.697</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Age</th>             <td> 4463.4461</td> <td> 1750.353</td> <td>    2.550</td> <td> 0.017</td> <td>  865.544</td> <td> 8061.348</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>YearsExperience</th> <td> 1098.8909</td> <td> 3298.652</td> <td>    0.333</td> <td> 0.742</td> <td>-5681.586</td> <td> 7879.368</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 1.673</td> <th>  Durbin-Watson:     </th> <td>   1.559</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.433</td> <th>  Jarque-Bera (JB):  </th> <td>   1.020</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td> 0.451</td> <th>  Prob(JB):          </th> <td>   0.601</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 3.058</td> <th>  Cond. No.          </th> <td>1.23e+03</td>\n",
       "</tr>\n",
       "</table><br/><br/>Notes:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.23e+03. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                 Salary   R-squared:                       0.966\n",
       "Model:                            OLS   Adj. R-squared:                  0.962\n",
       "Method:                 Least Squares   F-statistic:                     245.4\n",
       "Date:                Sat, 20 Jan 2024   Prob (F-statistic):           3.49e-19\n",
       "Time:                        16:33:15   Log-Likelihood:                -297.95\n",
       "No. Observations:                  30   AIC:                             603.9\n",
       "Df Residuals:                      26   BIC:                             609.5\n",
       "Df Model:                           3                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "===================================================================================\n",
       "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
       "-----------------------------------------------------------------------------------\n",
       "const           -6.833e+04   3.72e+04     -1.838      0.078   -1.45e+05    8094.672\n",
       "Sex              -311.4461   1984.473     -0.157      0.877   -4390.590    3767.697\n",
       "Age              4463.4461   1750.353      2.550      0.017     865.544    8061.348\n",
       "YearsExperience  1098.8909   3298.652      0.333      0.742   -5681.586    7879.368\n",
       "==============================================================================\n",
       "Omnibus:                        1.673   Durbin-Watson:                   1.559\n",
       "Prob(Omnibus):                  0.433   Jarque-Bera (JB):                1.020\n",
       "Skew:                           0.451   Prob(JB):                        0.601\n",
       "Kurtosis:                       3.058   Cond. No.                     1.23e+03\n",
       "==============================================================================\n",
       "\n",
       "Notes:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.23e+03. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.summary()#查看模型摘要，获取关键信息"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fd27dc5",
   "metadata": {},
   "source": [
    "## 应用案例（一）波士顿房价预测"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53ed6e22",
   "metadata": {},
   "source": [
    "### 1、导入函数库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "6b506303",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries \n",
    "import pandas as pd \n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt \n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eef794b",
   "metadata": {},
   "source": [
    "### 2、加载数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "1a2a049c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston#载入波士顿房价预测数据\n",
    " \n",
    "boston=load_boston()#加载数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e62e1537",
   "metadata": {},
   "source": [
    "### 3、划分数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "b57f8c71",
   "metadata": {},
   "outputs": [],
   "source": [
    "X=pd.DataFrame(boston['data'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d95a652",
   "metadata": {},
   "source": [
    "Bunch is a subclass of the Dict class and supports all the methods as dict does. In addition, it allows you to use the keys as attributes.\n",
    "\n",
    "Bunch是Dict类的子类，支持所有与dict相同的方法。此外，它还允许您将键用作属性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "3eadea0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "X.columns=boston['feature_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "fbe64a04",
   "metadata": {},
   "outputs": [],
   "source": [
    "#给特征集添加标签\n",
    "X.columns=boston['feature_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "9461ef16",
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00632</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.538</td>\n",
       "      <td>6.575</td>\n",
       "      <td>65.2</td>\n",
       "      <td>4.0900</td>\n",
       "      <td>1.0</td>\n",
       "      <td>296.0</td>\n",
       "      <td>15.3</td>\n",
       "      <td>396.90</td>\n",
       "      <td>4.98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.02731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>6.421</td>\n",
       "      <td>78.9</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.02729</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.07</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.469</td>\n",
       "      <td>7.185</td>\n",
       "      <td>61.1</td>\n",
       "      <td>4.9671</td>\n",
       "      <td>2.0</td>\n",
       "      <td>242.0</td>\n",
       "      <td>17.8</td>\n",
       "      <td>392.83</td>\n",
       "      <td>4.03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.03237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>6.998</td>\n",
       "      <td>45.8</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>394.63</td>\n",
       "      <td>2.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.06905</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.458</td>\n",
       "      <td>7.147</td>\n",
       "      <td>54.2</td>\n",
       "      <td>6.0622</td>\n",
       "      <td>3.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>18.7</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>0.06263</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.593</td>\n",
       "      <td>69.1</td>\n",
       "      <td>2.4786</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>391.99</td>\n",
       "      <td>9.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>0.04527</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.120</td>\n",
       "      <td>76.7</td>\n",
       "      <td>2.2875</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>9.08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>0.06076</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.976</td>\n",
       "      <td>91.0</td>\n",
       "      <td>2.1675</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>5.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>0.10959</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.794</td>\n",
       "      <td>89.3</td>\n",
       "      <td>2.3889</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>393.45</td>\n",
       "      <td>6.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>0.04741</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.573</td>\n",
       "      <td>6.030</td>\n",
       "      <td>80.8</td>\n",
       "      <td>2.5050</td>\n",
       "      <td>1.0</td>\n",
       "      <td>273.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>396.90</td>\n",
       "      <td>7.88</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>506 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        CRIM    ZN  INDUS  CHAS    NOX     RM   AGE     DIS  RAD    TAX  \\\n",
       "0    0.00632  18.0   2.31   0.0  0.538  6.575  65.2  4.0900  1.0  296.0   \n",
       "1    0.02731   0.0   7.07   0.0  0.469  6.421  78.9  4.9671  2.0  242.0   \n",
       "2    0.02729   0.0   7.07   0.0  0.469  7.185  61.1  4.9671  2.0  242.0   \n",
       "3    0.03237   0.0   2.18   0.0  0.458  6.998  45.8  6.0622  3.0  222.0   \n",
       "4    0.06905   0.0   2.18   0.0  0.458  7.147  54.2  6.0622  3.0  222.0   \n",
       "..       ...   ...    ...   ...    ...    ...   ...     ...  ...    ...   \n",
       "501  0.06263   0.0  11.93   0.0  0.573  6.593  69.1  2.4786  1.0  273.0   \n",
       "502  0.04527   0.0  11.93   0.0  0.573  6.120  76.7  2.2875  1.0  273.0   \n",
       "503  0.06076   0.0  11.93   0.0  0.573  6.976  91.0  2.1675  1.0  273.0   \n",
       "504  0.10959   0.0  11.93   0.0  0.573  6.794  89.3  2.3889  1.0  273.0   \n",
       "505  0.04741   0.0  11.93   0.0  0.573  6.030  80.8  2.5050  1.0  273.0   \n",
       "\n",
       "     PTRATIO       B  LSTAT  \n",
       "0       15.3  396.90   4.98  \n",
       "1       17.8  396.90   9.14  \n",
       "2       17.8  392.83   4.03  \n",
       "3       18.7  394.63   2.94  \n",
       "4       18.7  396.90   5.33  \n",
       "..       ...     ...    ...  \n",
       "501     21.0  391.99   9.67  \n",
       "502     21.0  396.90   9.08  \n",
       "503     21.0  396.90   5.64  \n",
       "504     21.0  393.45   6.48  \n",
       "505     21.0  396.90   7.88  \n",
       "\n",
       "[506 rows x 13 columns]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b6f39d5",
   "metadata": {},
   "source": [
    "### 4、寻找关联性强的特征"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5be7c5ca",
   "metadata": {},
   "source": [
    "#### (1)corr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "b1524059",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CRIM</th>\n",
       "      <th>ZN</th>\n",
       "      <th>INDUS</th>\n",
       "      <th>CHAS</th>\n",
       "      <th>NOX</th>\n",
       "      <th>RM</th>\n",
       "      <th>AGE</th>\n",
       "      <th>DIS</th>\n",
       "      <th>RAD</th>\n",
       "      <th>TAX</th>\n",
       "      <th>PTRATIO</th>\n",
       "      <th>B</th>\n",
       "      <th>LSTAT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>CRIM</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.200469</td>\n",
       "      <td>0.406583</td>\n",
       "      <td>-0.055892</td>\n",
       "      <td>0.420972</td>\n",
       "      <td>-0.219247</td>\n",
       "      <td>0.352734</td>\n",
       "      <td>-0.379670</td>\n",
       "      <td>0.625505</td>\n",
       "      <td>0.582764</td>\n",
       "      <td>0.289946</td>\n",
       "      <td>-0.385064</td>\n",
       "      <td>0.455621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZN</th>\n",
       "      <td>-0.200469</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.533828</td>\n",
       "      <td>-0.042697</td>\n",
       "      <td>-0.516604</td>\n",
       "      <td>0.311991</td>\n",
       "      <td>-0.569537</td>\n",
       "      <td>0.664408</td>\n",
       "      <td>-0.311948</td>\n",
       "      <td>-0.314563</td>\n",
       "      <td>-0.391679</td>\n",
       "      <td>0.175520</td>\n",
       "      <td>-0.412995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>INDUS</th>\n",
       "      <td>0.406583</td>\n",
       "      <td>-0.533828</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.062938</td>\n",
       "      <td>0.763651</td>\n",
       "      <td>-0.391676</td>\n",
       "      <td>0.644779</td>\n",
       "      <td>-0.708027</td>\n",
       "      <td>0.595129</td>\n",
       "      <td>0.720760</td>\n",
       "      <td>0.383248</td>\n",
       "      <td>-0.356977</td>\n",
       "      <td>0.603800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CHAS</th>\n",
       "      <td>-0.055892</td>\n",
       "      <td>-0.042697</td>\n",
       "      <td>0.062938</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.091203</td>\n",
       "      <td>0.091251</td>\n",
       "      <td>0.086518</td>\n",
       "      <td>-0.099176</td>\n",
       "      <td>-0.007368</td>\n",
       "      <td>-0.035587</td>\n",
       "      <td>-0.121515</td>\n",
       "      <td>0.048788</td>\n",
       "      <td>-0.053929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NOX</th>\n",
       "      <td>0.420972</td>\n",
       "      <td>-0.516604</td>\n",
       "      <td>0.763651</td>\n",
       "      <td>0.091203</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.302188</td>\n",
       "      <td>0.731470</td>\n",
       "      <td>-0.769230</td>\n",
       "      <td>0.611441</td>\n",
       "      <td>0.668023</td>\n",
       "      <td>0.188933</td>\n",
       "      <td>-0.380051</td>\n",
       "      <td>0.590879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RM</th>\n",
       "      <td>-0.219247</td>\n",
       "      <td>0.311991</td>\n",
       "      <td>-0.391676</td>\n",
       "      <td>0.091251</td>\n",
       "      <td>-0.302188</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.240265</td>\n",
       "      <td>0.205246</td>\n",
       "      <td>-0.209847</td>\n",
       "      <td>-0.292048</td>\n",
       "      <td>-0.355501</td>\n",
       "      <td>0.128069</td>\n",
       "      <td>-0.613808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AGE</th>\n",
       "      <td>0.352734</td>\n",
       "      <td>-0.569537</td>\n",
       "      <td>0.644779</td>\n",
       "      <td>0.086518</td>\n",
       "      <td>0.731470</td>\n",
       "      <td>-0.240265</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.747881</td>\n",
       "      <td>0.456022</td>\n",
       "      <td>0.506456</td>\n",
       "      <td>0.261515</td>\n",
       "      <td>-0.273534</td>\n",
       "      <td>0.602339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DIS</th>\n",
       "      <td>-0.379670</td>\n",
       "      <td>0.664408</td>\n",
       "      <td>-0.708027</td>\n",
       "      <td>-0.099176</td>\n",
       "      <td>-0.769230</td>\n",
       "      <td>0.205246</td>\n",
       "      <td>-0.747881</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.494588</td>\n",
       "      <td>-0.534432</td>\n",
       "      <td>-0.232471</td>\n",
       "      <td>0.291512</td>\n",
       "      <td>-0.496996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RAD</th>\n",
       "      <td>0.625505</td>\n",
       "      <td>-0.311948</td>\n",
       "      <td>0.595129</td>\n",
       "      <td>-0.007368</td>\n",
       "      <td>0.611441</td>\n",
       "      <td>-0.209847</td>\n",
       "      <td>0.456022</td>\n",
       "      <td>-0.494588</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.910228</td>\n",
       "      <td>0.464741</td>\n",
       "      <td>-0.444413</td>\n",
       "      <td>0.488676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TAX</th>\n",
       "      <td>0.582764</td>\n",
       "      <td>-0.314563</td>\n",
       "      <td>0.720760</td>\n",
       "      <td>-0.035587</td>\n",
       "      <td>0.668023</td>\n",
       "      <td>-0.292048</td>\n",
       "      <td>0.506456</td>\n",
       "      <td>-0.534432</td>\n",
       "      <td>0.910228</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.460853</td>\n",
       "      <td>-0.441808</td>\n",
       "      <td>0.543993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PTRATIO</th>\n",
       "      <td>0.289946</td>\n",
       "      <td>-0.391679</td>\n",
       "      <td>0.383248</td>\n",
       "      <td>-0.121515</td>\n",
       "      <td>0.188933</td>\n",
       "      <td>-0.355501</td>\n",
       "      <td>0.261515</td>\n",
       "      <td>-0.232471</td>\n",
       "      <td>0.464741</td>\n",
       "      <td>0.460853</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.177383</td>\n",
       "      <td>0.374044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>B</th>\n",
       "      <td>-0.385064</td>\n",
       "      <td>0.175520</td>\n",
       "      <td>-0.356977</td>\n",
       "      <td>0.048788</td>\n",
       "      <td>-0.380051</td>\n",
       "      <td>0.128069</td>\n",
       "      <td>-0.273534</td>\n",
       "      <td>0.291512</td>\n",
       "      <td>-0.444413</td>\n",
       "      <td>-0.441808</td>\n",
       "      <td>-0.177383</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.366087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LSTAT</th>\n",
       "      <td>0.455621</td>\n",
       "      <td>-0.412995</td>\n",
       "      <td>0.603800</td>\n",
       "      <td>-0.053929</td>\n",
       "      <td>0.590879</td>\n",
       "      <td>-0.613808</td>\n",
       "      <td>0.602339</td>\n",
       "      <td>-0.496996</td>\n",
       "      <td>0.488676</td>\n",
       "      <td>0.543993</td>\n",
       "      <td>0.374044</td>\n",
       "      <td>-0.366087</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             CRIM        ZN     INDUS      CHAS       NOX        RM       AGE  \\\n",
       "CRIM     1.000000 -0.200469  0.406583 -0.055892  0.420972 -0.219247  0.352734   \n",
       "ZN      -0.200469  1.000000 -0.533828 -0.042697 -0.516604  0.311991 -0.569537   \n",
       "INDUS    0.406583 -0.533828  1.000000  0.062938  0.763651 -0.391676  0.644779   \n",
       "CHAS    -0.055892 -0.042697  0.062938  1.000000  0.091203  0.091251  0.086518   \n",
       "NOX      0.420972 -0.516604  0.763651  0.091203  1.000000 -0.302188  0.731470   \n",
       "RM      -0.219247  0.311991 -0.391676  0.091251 -0.302188  1.000000 -0.240265   \n",
       "AGE      0.352734 -0.569537  0.644779  0.086518  0.731470 -0.240265  1.000000   \n",
       "DIS     -0.379670  0.664408 -0.708027 -0.099176 -0.769230  0.205246 -0.747881   \n",
       "RAD      0.625505 -0.311948  0.595129 -0.007368  0.611441 -0.209847  0.456022   \n",
       "TAX      0.582764 -0.314563  0.720760 -0.035587  0.668023 -0.292048  0.506456   \n",
       "PTRATIO  0.289946 -0.391679  0.383248 -0.121515  0.188933 -0.355501  0.261515   \n",
       "B       -0.385064  0.175520 -0.356977  0.048788 -0.380051  0.128069 -0.273534   \n",
       "LSTAT    0.455621 -0.412995  0.603800 -0.053929  0.590879 -0.613808  0.602339   \n",
       "\n",
       "              DIS       RAD       TAX   PTRATIO         B     LSTAT  \n",
       "CRIM    -0.379670  0.625505  0.582764  0.289946 -0.385064  0.455621  \n",
       "ZN       0.664408 -0.311948 -0.314563 -0.391679  0.175520 -0.412995  \n",
       "INDUS   -0.708027  0.595129  0.720760  0.383248 -0.356977  0.603800  \n",
       "CHAS    -0.099176 -0.007368 -0.035587 -0.121515  0.048788 -0.053929  \n",
       "NOX     -0.769230  0.611441  0.668023  0.188933 -0.380051  0.590879  \n",
       "RM       0.205246 -0.209847 -0.292048 -0.355501  0.128069 -0.613808  \n",
       "AGE     -0.747881  0.456022  0.506456  0.261515 -0.273534  0.602339  \n",
       "DIS      1.000000 -0.494588 -0.534432 -0.232471  0.291512 -0.496996  \n",
       "RAD     -0.494588  1.000000  0.910228  0.464741 -0.444413  0.488676  \n",
       "TAX     -0.534432  0.910228  1.000000  0.460853 -0.441808  0.543993  \n",
       "PTRATIO -0.232471  0.464741  0.460853  1.000000 -0.177383  0.374044  \n",
       "B        0.291512 -0.444413 -0.441808 -0.177383  1.000000 -0.366087  \n",
       "LSTAT   -0.496996  0.488676  0.543993  0.374044 -0.366087  1.000000  "
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd0a99c9",
   "metadata": {},
   "source": [
    "数据列太多了，肉眼难以辨别"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9aea617d",
   "metadata": {},
   "source": [
    "#### （2）绘制热力图"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "b6da3934",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi4AAAHWCAYAAABQYwI2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4MklEQVR4nO3deZwsZXm38evnAQTDQeKGCCpukSiILG5xAxFFg+LOorJEPZq4ghu+vhqNETVGQYKKJ7iAMYp53RAJGBXQuHIQZBVE3NhFUDCggHO/f3SdoRlmumbpnu5OX9/zqc/pWrrqN93TM/c8z1NVqSokSZLGwe2GHUCSJGm+LFwkSdLYsHCRJEljw8JFkiSNDQsXSZI0NixcJEnS2LBwkSRJC5bkY0muTHL2HOuT5LAkFyY5M8l2/TiuhYskSVqMTwC79lj/FOABzbQK+HA/DmrhIkmSFqyqvglc3WOT3YGjq+N7wMZJNl3qcS1cJEnSIGwG/Kpr/uJm2ZKss9QdtLnpqotG5p4C79z+LcOOAMABT7hi2BEA+PKJdx92hGlP/+srhx0BgPf8112GHQGA1257ybAjTHvpDzcedgQAjnrTFsOO0HH99cNOMO26Y38y7AgAvPxnGw07AgAf331q2BGmrTzsuCzn8Qbxu3a9u97vpXS6eNZaXVWrF7CL2V6DJecceOEiSZLGT1OkLKRQmeli4J5d85sDly4pFBYukiSNv6k/DTvBbI4FXpHkM8AjgN9V1WVL3amFiyRJ466Wv5ssyaeBHYG7JLkY+HtgXYCqOgI4HngqcCFwPbB/P45r4SJJkhasqvZqWV/Ay/t9XAsXSZLG3dToDEweNE+HliRJY6Nni0uSZ/VaX1Wf728cSZK0UDWEMS7D0tZV9P+AM5oJbn1OdgEWLpIkDdsEdRW1FS7PBvYAHgJ8Cfh0VV048FSSJEmz6DnGpaq+UFV7Ao8Hfgq8L8l/J3n8sqSTJEntaqr/04ia7+DcPwC/A64F/gxYf2CJJEmS5tA2OHcnYC/g4cDXgA9U1ZrlCCZJkuZpNK+cOxBtY1y+DpwJ/Ddwe2CfJPusXVlVrxpgNkmSpFtpK1z+hj7cyVGSJA3QCI9J6beehUtVfWKZckiSpMXydOiOJF+mR4tLVT2974kkSZLm0NZV9M+L2WmSVcAqgA+97x958T4978MkSZKWwCvnNqrqlLnWJXl0j+etBlYD3HTVRY6RkSRJfdHWVbQCeB6wGXBCVZ2dZDfg/wAbANsOPqIkSerJMS7TPgrcE/gBcFiSXwCPAg6qqi8OOJskSZoPu4qm7QA8pKqmkqwPXAXcv6ouH3w0SZKkW2srXG6sZsRPVf0hyQUWLZIkjRivnDttyyRnNo8D3K9rnqp6yMCSSZIkzdBWuGwDbAL8asbyewOXDiSRJElamAka49J2d+hDgGur6hfdE3B9s06SJA3b1FT/pxHVVrhsUVVnzlzY3CF6i4EkkiRJmkNbV9H6PdZt0M8gkiRpkewqmnZqkpfMXJjkRcBpg4kkSZI0u7YWl9cAX0jyfG4pVHYA1gOeOcBckiRpvkZ4TEq/td2r6Argr5LsBGzVLP5KVX1j4MkkSdK8VHkdl1upqpOAkwacRZIkqad5FS6SJGmETdDg3IEXLu/c/i2DPsS8vfm0dww7AgCP2nrfYUcA4LWpYUeY9sSv/HHYEQD4yoMvGXYEAB77nT8MO8K0Ne990LAjAPCqt1447AgAFKPzubm6Nhp2BADeud7Nw44AwN5fbDvfZPl8+bBhJ/jfyxYXSZLG3QQNzh2d8lSSJKmFLS6SJI07x7hIkqSxMTU5p0PbVSRJksaGLS6SJI27CeoqssVFkiSNDVtcJEkadxN0OrSFiyRJ486uIkmSpNFji4skSeNugrqKbHGRJEmLkmTXJOcnuTDJQbOsv2OSLyf5UZJzkuy/1GP2bHFJsk+v9VV19FIDSJKkJRpCi0uSFcAHgV2Ai4FTkxxbVed2bfZy4NyqelqSuwLnJ/lUVd242OO2dRU9bLaswNOAzQALF0mShqxqKFfOfThwYVVdBJDkM8DuQHfhUsDKJAE2BK4GlnQ78Z6FS1W9cu3j5qDPB94IfA9451IOLEmSxtpmwK+65i8GHjFjm8OBY4FLgZXAHlVLOwWqdXBuknWA/YDXAt8HnlNV5y/loJIkqY8G0FWUZBWwqmvR6qpa3b3JLE+rGfNPBs4AngDcD/ivJN+qqmsXm6vn4NwkL6fT5LM9sGtV7TefoiXJqiRrkqw57fcXLjabJEkakqpaXVU7dE2rZ2xyMXDPrvnN6bSsdNsf+Hx1XAj8DNhyKbnazir6F2Aj4DHAl5Oc2UxnJTlzrid1f7Hbb3j/peSTJEltaqr/U7tTgQckuU+S9YA96XQLdfslsDNAkk2ABwIXLeVLbesq+jvgBG7b9AOwx1IOLEmSxldV3ZzkFcCJwArgY1V1TpKXNeuPAN4BfCLJWXS6lt5YVVct5bhthcsHgVOAF1bVJd0rkuwJ/NNSDi5JkvpgSBegq6rjgeNnLDui6/GlwJP6ecy2rqIzgU8D30vy3BnrZhuUI0mSlttwuoqGoq1wqar6Vzr9U29I8vEkd1i7brDRJEmSbm1el/yvqguARwFXAKcnmXmetiRJGpapqf5PI6ptjMt0d1BV3QwclOQEOt1Hdx1kMEmSpJnaCpe3z1xQVScn2R546WAiSZKkBRnhMSn91nbJ/y/Osfwa4N2DCCRJkhZohLt2+m1eY1wkSZJGQeu9iiRJ0oizxUWSJGn02OIiSdK4c3CuJEkaG3YVSZIkjR5bXCRJGnd2FfXPAU+4YtCHmLdHbb3vsCMA8N2zjhp2BAC+/eA3DjvCtFOOnnkPz+F4/qqvDjsCAGsOftiwI0zb5G+PGXYEAC4/uK83mF203GuLYUeY9sfPnjjsCAD83Xc2HnYEAI55/abDjqBlYIuLJEnjzjEukiRJo2dBLS5J1gW2Ai6pqisHE0mSJC3IBI1x6dnikuSIJA9uHt8R+BFwNHB6kr2WIZ8kSWozNdX/aUS1dRU9tqrOaR7vD1xQVVsD2wNvGGgySZKkGdq6im7serwL8B8AVXV5koGFkiRJCzDCLST91tbi8tskuyXZFng0cAJAknWADQYdTpIkqVtbi8tLgcOAuwOvqarLm+U7A18ZZDBJkjRPVcNOsGx6Fi5VdQGw6yzLTwRG48pHkiRNugnqKupZuCT5F6C7jCvgKuCkqvrvQQaTJEmaqa2raM0sy+4EvDfJMVV1aP8jSZKkBbHFpaOqZr2pTpIjgO8Ahw4gkyRJ0qwWda+iqrrB06ElSRoRE3Tl3AUXLs2p0C8ELu5/HEmStGB2FXUkuY7OgNzu5pXrgVPonCotSZK0bNrGuKxcriCSJGmRJug6Lm1XziXJOkmeluT1zfTXTXdRr+esSrImyZpPnH9J/9JKkqSJ1laA3AM4CbgMOJ1Ol9FuwPuT7FRVl872vKpaDawG+N3+T5ycMlCSpGFwjMu0g4EPz7xeS5JXAe8C9h1QLkmSpNtoK1weWVX7zVxYVYclOX8wkSRJ0oLY4jLthh7rru9nEEmStEhex2XaHZM8a5blATYaQB5JkqQ5tRUupwBPm2PdN/ucRZIkLUJNTc55MG3Xcdl/uYJIkiS1aTsdep8eq6uqPtnnPJIkaaEcnDvtYbMsC53uo80ACxdJkobNwbkdVfXKtY/TuR3084E3At8D3jnYaJIkSbc230v+vxg4F3gi8Jyq2qOqzhx4OkmS1G6q+j/NQ5Jdk5yf5MIkB82xzY5JzkhyTpJTlvqlto1xeTnwauDrwK5V9YulHlCSJI2/JCuADwK7ABcDpyY5tqrO7dpmY+BDdGqIXya521KP2zbG5V+AK4HHAF/u9BZ1stAZnPuQpQaQJElLNJzBuQ8HLqyqiwCSfAbYnU4PzVp7A5+vql8CVNWVSz1oW+Fyn6UeQJIkDdgACpckq4BVXYtWNzdRXmsz4Fdd8xcDj5ixm78A1k1yMrAS+EBVHb2UXG2Dc+0akiRpAjVFyuoem2SWZTMHx6wDbA/sDGwAfDfJ96rqgsXmStXcA3CSXDdLCLilq6j1sv//do8XjMzl/FbM+qUsv03/dOOwIwDw6HPeM+wI007d+vXDjjBSqmb7eTAcZ627/rAjAHD3m0bjdM8/ZnTemyvXGY0sD7rxpmFHAOCyFesOO8K0F1z6b8v65lx/6Ev7/gvuDq/5SM+vIcmjgLdV1ZOb+TcBVNW7urY5CFi/qt7WzH8UOKGq/mOxuXqeVVRVK6tqo1mmlfMpWiRJ0v9apwIPSHKfJOsBewLHztjmS8BjmzOU70CnK+m8pRy0bYyLJEkadUMYnFtVNyd5BXAisAL4WFWdk+Rlzfojquq8JCcAZwJTwJFVdfZSjmvhIkmSFqWqjgeOn7HsiBnz7wXe269jWrhIkjTuvDu0JEkaGxN0r6Keg3OTvCTJA5rHSfLxJNcmOTPJdssTUZIkqaPtXkWvBn7ePN4LeAidi9IdCHxgcLEkSdK8DeleRcPQVrjcXFVrT9DfDTi6qn5TVV8D/myw0SRJkm6tbYzLVJJNgWvoXPXunV3rNhhYKkmSNG81nHsVDUVb4fJWYA2d87OPrapzAJI8HrhowNkkSdJ8jHDXTr+13avouCT3BlZW1TVdq06lc4U8SZKkZdN6OnRV3Uynq4gkAXaic5vqpwGbDDSdJElq5+nQt5bkEUk+APyCzn0IvgVsOchgkiRJM7Vdx+WdSX4CHAycBWwL/LqqjprRdSRJkoZlgk6HbusqWgWcD3wYOK6q/pBkdL8aSZIm0QSdVdTWVXR3OqdAPx24MMkngQ2SeKsASZK07HoWLlX1p6r6z6raB7g/8CXgO8AlSf59ruclWZVkTZI137j+J/1NLEmSbm2CuormNTgXoKr+UFX/r6qeTaeIObHHtquraoeq2uEJd3hAP3JKkiT1HuOS5MDlCiJJkhbJ06GnreyaXjdjfsPBRpMkSbq1tivnvn3t4yTP6J6XJEkjYoTHpPTbQs4OmpxXRZKkMTJJN1mc9+BcSZKkYWsbnHsWt7S03D/Jmd3rq+ohgwomSZLmya6iac+icyPFX81Yfm/g0oEkkiRJmkNbV9EhwLVV9YvuCbi+WSdJkoZtgi5A19biskVVnTlzYVWtSbLFYCJJkqQF8Tou09bvsW6DfgaRJElq01a4nJrkJTMXJnkRcNpgIkmSpAWxq2jaa4AvJHk+txQqOwDrAc8cYC5JkqTbaLty7hXAXyXZCdiqWfyVqvrGwJNJkqR5qRFuIem3eV05t6pOAk4acBZJkrQYE1S4eOVcSZI0NhZyr6JFefpfXznoQ8zbE7/yx2FHAOCUo5877AgAnLr164cdYdrDznrvsCMA8Iod3jjsCAAcetgjhh1h2uOfMxqXbLr2kNEYVpeVK4cdYdovDz5j2BEA+L+3W2/YEQA46oCNhx1heLxXkSRJ0ugZeIuLJEkaMMe4SJIkjR5bXCRJGncT1OJi4SJJ0pirmpzCxa4iSZI0Nhbd4pJknaq6uZ9hJEnSIkxQV1HPFpckX05y71mWPxE4Y1ChJEmSZtPW4vIZ4KQkHwX+CbgrcChwL2DfwUaTJEnzYotLR1V9CtiWTqFyHvBd4GvAo6rqtF7PlSRJy6Omqu/TfCTZNcn5SS5MclCP7R6W5E9JnrPUr3U+g3MfBDwc+AHwR2ATPBtJkqSJlmQF8EHgKXRqhb2SPGiO7d4DnNiP47aNcTmyCfV3VbU3ndaXOwI/SvKkfgSQJElLNFX9n9o9HLiwqi6qqhvpDC/ZfZbtXgl8DujLzQvbWlzOAR5WVd8FqKr/qarXAXsAb+lHAEmSNHqSrEqypmtaNWOTzYBfdc1f3Czr3sdmwDOBI/qVq2eXT1UdkuRuSV4OPBgo4FzgQ1X12H6FkCRJSzCAm0NX1WpgdY9NMtvTZswfCryxqv6UzLb5wvUsXJI8Gvh34BPA0XRCbgd8P8nzq+rbfUkhSZIWbb6DafvsYuCeXfObA5fO2GYH4DNN0XIX4KlJbq6qLy72oG2DbN8HPKOqTu9a9qUkXwA+Ajxitic1zUmrAA59zF+y/5abLzafJEkaTacCD0hyH+ASYE9g7+4Nquo+ax8n+QRw3FKKFmgvXDaaUbSsDXJGkpVzPam7eenalzxpck4ulyRpGIbQ4lJVNyd5BZ2zhVYAH6uqc5K8rFnft3Et3doKlyT586q6ZsbCO+F9jiRJmmhVdTxw/IxlsxYsVbVfP47ZVnwcAnw1yeOTrGymHYH/bNZJkqRhmxrANKLazipaneRS4B10ziqCzinS/1hVXx50OEmSpG6tV8CtquOA45YhiyRJWoQhnVU0FG2nQ7+1x+qqqnf0OY8kSVqoEe7a6be2Fpf/mWXZnwEvAu5MpwtJkiRpWbSNcXnf2sfN6c+vBvancz+C9831PEmStHzsKurSnPp8IPB84Chgu5mnR0uSJC2HtjEu7wWeRedicltX1e+XJZUkSZo/x7hMey3wR+D/Am/uukFS6AzO3WiA2SRJ0jyUhUtHVXl1XEmSNDJax7hIkqQRN0EtLraoSJKksWGLiyRJY84xLpIkaXxMUOGSqsFetObNW+w9MlfFec0DLhl2BABeesEdhx0BgNfduO6wI0w76vaj0Wt5+Jr3DDsCAG/e4c3DjjDtpRtePewIAHz893cedgQArubmYUeY9tu6adgRAHjd1Gj81nzODb8cdoRpF111etq36p+rnvz4vv+uvcuJpyzr1zBftrhIkjTmJqmraDT+zJUkSZoHW1wkSRpzk9TiYuEiSdKYm6TCxa4iSZI0NmxxkSRp3NVIngA0ELa4SJKksWGLiyRJY84xLpIkSSPIFhdJksZcTU3OGBcLF0mSxtwkdRX1LFySHNZrfVW9qr9xJEmS5tbW4vIy4Gzgs8ClwOS0RUmSNCZqgk6HbitcNgWeC+wB3AwcA3yuqq4ZdDBJkqSZep5VVFW/qaojqmonYD9gY+CcJC9chmySJGkeaqr/06ia1+DcJNsBewG7AP8JnDbIUJIkaf4m6ayini0uSd6e5DTgQOAUYIeqelFVndvyvFVJ1iRZc/p1F/YxriRJmmRtF6B7C3BHYBvgXcAPk5yZ5KwkZ871pKpaXVU7VNUO2668fx/jSpKkmar6P42qtq6i+yxLCkmSpHnoWbhU1S9mW55kBbAnMOt6SZK0fBzj0kiyUZI3JTk8yZPS8UrgIuB5yxNRkiT1UlPp+zSq2rqKPglcA3wXeDHwemA9YPeqOmOw0SRJkm6trXC5b1VtDZDkSOAq4F5Vdd3Ak0mSpHkZ5cG0/dZ2VtFNax9U1Z+An1m0SJKkYWlrcdkmybXN4wAbNPMBqqo2Gmg6SZLUapTHpPRb2yX/V1TVRs20sqrW6Xps0SJJ0gRLsmuS85NcmOSgWdY/v7n+25lJvpNkm6Uec16X/JckSaNrGHeHbi6N8kE6twO6GDg1ybEzrq7/M+DxVXVNkqcAq4FHLOW4Fi6SJI25Id0U8eHAhVV1EUCSzwC7A9OFS1V9p2v77wGbL/WgbYNzJUmSZrMZ8Kuu+YubZXN5EZ0bNS+JLS6SJI25qQF0FSVZBazqWrS6qlZ3bzLL02Y9MTvJTnQKl8csNZeFiyRJuo2mSFndY5OLgXt2zW8OXDpzoyQPAY4EnlJVv1lqLgsXSZLG3DAG5wKnAg9Ich/gEjr3MNy7e4Mk9wI+D7ywqi7ox0EtXCRJGnPDuI5LVd2c5BXAicAK4GNVdU6SlzXrjwDeCtwZ+FASgJuraoelHHfghctrt71k0IeYt8d+5w/DjgDAmoMfNuwIAPzwzT8fdoRphx62pLPj+ubNO7x52BEAeOeadw47wrQNN3/8sCMAcNWqJV/+oS9ut3KDYUeY9ttv/G7YEQB4z2V3HXYEAM7++0cNO8LEqarjgeNnLDui6/GL6dzrsG9scZEkacx5ryJJkqQRZIuLJEljbpLuVWThIknSmBvEdVxGlV1FkiRpbNjiIknSmBvSdVyGwhYXSZI0NuYsXJJs1GPdvQYTR5IkLVRV/6dR1avF5eS1D5J8fca6Lw4ijCRJUi+9xrh0d5jdqcc6SZI0RJN0VlGvwqXmeDzbvCRJGpJJGpzbq3C5W5ID6bSurH1MMz8aN6aQJEkTpVfh8q/AylkeAxw5sESSJGlBRnkwbb/NWbhU1duXM4gkSVKbOQuXJA8G7ldVxzbzhwB3bFYfXlU/XIZ8kiSpxSQNzu11OvS7gau65p8MfAU4CXhrr50mWZVkTZI1R/38sqWnlCRJc6pK36dR1WuMy6ZV9Z2u+Wur6nMASV7aa6dVtRpYDXD1Mx8/QT1vkiRpkHoVLt2DcamqR3bN3m0wcSRJ0kLZVdRxaZJHzFyY5JHApYOLJEmSNLteLS5vBI5J8glg7UDc7YF9gT0GnEuSJM3TJI3JmLPFpap+ADwSWAHsR6dgCfDC5rEkSRoBU5W+T6OqV4sLVXUF8NYk2wJ70SlYHgd8bhmySZIk3Uqv67j8BbAnnYLlN8AxQKpqp2XKJkmS5mGUT1/ut14tLj8GvgU8raouBEhywLKkkiRJmkWvs4qeDVwOnJTkX5PsTGeMiyRJGiFTA5hGVa/BuV+oqj2ALYGTgQOATZJ8OMmTlimfJEnStF4tLgBU1f9U1aeqajdgc+AM4KBBB5MkSfNTpO/TqOp5VtFMVXU18JFmkiRJI2Bqgi7k0triIkmSNCoW1OIiSZJGz9QId+30my0ukiRpbNjiIknSmBvlwbT9NvDC5aU/3HjQh5i3Ne990LAjALDJ3x4z7AgA/NOd/mrYEaY9/jmHDDsCAOf/xVbDjgDAhps/ftgRpv3+4lOGHQGAHbd58bAjAHDD1G+HHWHaLre/57AjAPDmLS4bdgQAdnn3r4cdYdq3X7m8xxvl6670m11FkiRpbNhVJEnSmJukriJbXCRJ0tiwxUWSpDE3SWNcLFwkSRpzk1S42FUkSZLGhi0ukiSNOQfnSpIktUiya5Lzk1yY5KBZ1ifJYc36M5Nst9Rj2uIiSdKYmxpCg0uSFcAHgV2Ai4FTkxxbVed2bfYU4AHN9Ajgw83/i2aLiyRJWoyHAxdW1UVVdSPwGWD3GdvsDhxdHd8DNk6y6VIOuqDCJcmdkzwzyfZLOagkSeqfKdL3aR42A37VNX9xs2yh2yxIz8IlyXFJtmoebwqcDfwN8Mkkr1nKgSVJUn/UAKYkq5Ks6ZpWzTjsbNVNLWKbBWkb43Kfqjq7ebw/8F9VtU+SlcC3gUOXcnBJkjSaqmo1sLrHJhcD3Xf63By4dBHbLEhbV9FNXY93Bo4HqKrrmKzr3UiSNLKmBjDNw6nAA5LcJ8l6wJ7AsTO2ORbYpzm76JHA76pqSbcTb2tx+VWSV9KpmLYDTgBIsgGw7lIOLEmSxldV3ZzkFcCJwArgY1V1TpKXNeuPoNPg8VTgQuB6Or03S9JWuLwI+AfgicAeVfXbZvkjgY8v9eCSJGnppjKcC9BV1fE0vTFdy47oelzAy/t5zJ6FS1VdCbxsluUnASfN9bxmAM8qgO3u9BDuu+EWS0spSZLmtKTRrmOmZ+GS5Mv0eD2q6ulzLJ8e0PPce+8+Sa+nJEkaoLauon9elhSSJGnRJulsmbauolPWPk5y12bZrwcdSpIkaTatV85N8vdJrgJ+DFyQ5NdJ3jr4aJIkaT6m0v9pVLVdOfcA4DHAw6rqzlX153RujvToZp0kSRqyIV3yfyjaWlz2Afaqqp+tXVBVFwEvaNZJkiQtm7bBuetW1VUzF1bVr5N4ATpJkkbAJJ2+29bicuMi10mSJPVdW4vLNkmunWV5gPUHkEeSJC3QKA+m7be206FXLFcQSZKkNm0tLpIkacR5ATpJkjQ2HJwrSZI0gmxxkSRpzE3S4FxbXCRJ0tiwxUWSpDHn4Nw+OupNWwz6EPP2qrdeOOwIAFx+8JOGHQGAr75rtkv0DMe1hzxz2BEAOPi9Vw87AgBXrdpm2BGm7bjNi4cdAYCTf3TksCMAMHX1pcOOMO3GD71n2BEAePbnRuOyXie+4A7DjjA0k1S42FUkSZLGhl1FkiSNuXJwriRJ0uixxUWSpDE3SWNcLFwkSRpzk1S42FUkSZLGhi0ukiSNuUm6V1Fr4ZJkHeApwJbNovOAE6rq5kEGkyRJmqln4ZLkHsBJwGXA6UCA3YD3JdmpqkbnSkySJE2oSbpXUVuLy8HAh6vq0O6FSV4FvAvYd0C5JEmSbqOtcHlkVe03c2FVHZbk/MFEkiRJCzFJZxW1FS439Fh3fT+DSJKkxbFwucUdkzxrluUBNhpAHkmSpDm1FS6nAE+bY903+5xFkiQtgqdDN6pq/7nWJdmk/3EkSZLmtqAr5ya5Y5K/SfI14Ic9tluVZE2SNR/71tlLDilJkuY2lf5Po2o+F6DbAHg6sDewHbASeAY9uoqqajWwGuD6I149SS1YkiQtu0kanNuzxSXJp4ALgCcBhwNbANdU1clVNUmvkyRJGgFtLS5bAdfQucz/j6vqT0lsQZEkaYRM0i/mni0uVbUN8Dw6pz5/Lcm3gJVJ7r4c4SRJkrq1jnGpqh8DbwXemmQHOmNdfpDk4qr6q0EHlCRJvU1NUJtLa+HSrarWAGuSHESnJUaSJA3ZJA06bRucu1GSNyU5PMmT0vEKOgN2n7s8ESVJ0jhJcqck/5XkJ83/fz7LNvdMclKS85Kck+TV89l323VcPgk8EDgLeDHwVToFyzOqavcFfh2SJGkAagDTEh0EfL2qHgB8vZmf6WbgtVX1l8AjgZcneVDbjtu6iu5bVVsDJDkSuAq4V1Vdt5D0kiRpouwO7Ng8Pgo4GXhj9wZVdRlwWfP4uiTnAZsB5/bacVvhclPXAf6U5GcWLZIkjZYRHOOySVOYUFWXJblbr42TbAFsC3y/bcdthcs2Sa5du19gg2Y+nSzlHaIlSfpfKMkqYFXXotXNlfHXrv8aMNvlUd68wONsCHwOeE1VXdu2fdtNFlcs5OCSJGn5DeLeQt2375lj/RPnWpfkiiSbNq0tmwJXzrHdunSKlk9V1efnk2tBN1mUJEmjZ4rq+7RExwL7No/3Bb40c4MkAT4KnFdV75/vji1cJElSv70b2CXJT4BdmnmS3CPJ8c02jwZeCDwhyRnN9NS2HS/oAnSSJGn0jNp1c6vqN8DOsyy/FHhq8/i/6YyZXRBbXCRJ0tgYfIvL9dcP/BDzVSNSk+ZeWww7AgB/zFnDjjAtK1cOOwIAV88+fmzZ3W7lBsOOMO2Gqd8OOwIAU1dfOuwIANzuTvcYdoRbTI3Iz7SF/9E8ENlow2FHGJoRPB16YOwqkiRpzE3STRbtKpIkSWPDFhdJksbc5LS32OIiSZLGiC0ukiSNOQfnSpKkseHgXEmSpBE0Z+GS5GE91r1wMHEkSdJC1QCmUdWrxeWjST6cZOO1C5JsleSbwLMHnkySJGmGXmNctgNeD5ye5B3A1nTuL/DaqjpuOcJJkqR2Ds4Fqupm4F1JbgaOBC4FHt7cIEmSJI2IUbmlzXLoNcblfklOBHYC/hL4Z+CbSfZfrnCSJEndeo1xORH416p6alWdX1WHAo8Dnpzk28uSTpIktZoawDSqeo1xeWhV/b57QdNNtGeSJw42liRJ0m31GuPy+5nLktwP2AvYE9hqgLkkSdI8eQG6Lkk2TfKaJD8AzgFW0CleJEmSllWvwbkvSfIN4BTgLsCLgcuq6u1VdVavnSZZlWRNkjUf++6P+5tYkiTdyiRdgK7XGJcPAt8F9q6qNQBJ5vW1VNVqYDXA9e9/ySh//ZIkjb1J6irqVbhsTucKue9PsgnwWWDdZUklSZI0i15jXE6oqg9X1eOAnYHfAVcmOS/JwcsTT5IktZmk06F7FS5Z+6CqLq6qf66q7YHdgT8OPJkkSdIMvbqK7prkwDnWXTeIMJIkaeEm6ZL/vQqXFcCGdLW8dJmcV0iSpBE3yl07/darcLmsqv5h2ZJIkiS16FW4zNbSIkmSRswkdRX1Gpy787KlkCRJmode9yq6ejmDSJKkxXGMiyRJGhtTZVeRJEnSyLHFRZKkMTc57S22uEiSpDHS2uKS5G7Ay4EH0ynqzgU+VFVXDDibJEmaB+8O3UjyaODfgU8AR9O5tst2wPeTPL+qvt12gOuO/UkfYvbH1bXRsCMA8MfPnjjsCABcuc5mw44w7ZcHnzHsCAD8ttYfdgQAfvuN3w07wrRdbn/PYUcA4MYPvWfYETqmRucXxPpvPWzYEQB43JfeMuwIANx05q+GHUHLoK3F5X3AM6rq9K5lX0ryBeAjwCMGlkySJM3LJF2Arq1w2WhG0QJAVZ2RZOWAMkmSpAWYpOu4tA3OTZI/n2XhnebxXEmSpL5qKz4OAb6a5PFJVjbTjsB/NuskSdKQTVF9n0ZVz66iqlqd5FLgHXTOKgI4B/jHqvryoMNJkiR1az0duqqOA45bhiySJGkRRm1wbjOk5BhgC+DnwPOq6po5tl0BrAEuqard2vbddjr0v9DjgnxV9aq2A0iSpMEawcG5BwFfr6p3JzmomX/jHNu+GjgPmNc1S9paXNbMO6IkSVLH7sCOzeOjgJOZpXBJsjnw18A7gQPns+O2wuWBVfV/5ptSkiQtvxq9u0NvUlWXAVTVZc1V+GdzKPAGYN6XWGk7q2jX+e5IkiT975FkVZI1XdOqGeu/luTsWabd57n/3YArq+q0heRqa3FZ0VzHJbOtrKqrF3IwSZLUf4M4fbmqVgOre6x/4lzrklyRZNOmtWVT4MpZNns08PQkTwXWBzZK8m9V9YJeudoKly2B05i9cCngvi3PlyRJAzaCg3OPBfYF3t38/6WZG1TVm4A3ATTXiHtdW9EC7YXLuVW17QLDSpKkyfZu4LNJXgT8EnguQJJ7AEdW1VMXu+PW67jMJckmVXXFYp8vSZL6Y9Su41JVvwF2nmX5pcBtipaqOpnOmUet2gbnfqB7Jskdk/xNkq8BP5zPASRJkvql7ZL/n0iyAfB0YG9gOzqnLD0D+ObA00mSpFajfG+hfuvZ4pLkU8AFwJOAw+lcuveaqjq5quYcC9R9CtUnL720n3klSdIEaxvjshVwDZ1L8f64qv6UpLWs6z6F6oodd5ycMlCSpCEYwQvQDUxbV9E2Sbak0030tSRXAiuT3L2qLl+WhJIkqacRPB16YNoG51JVP66qt1bVA4EDgE8CP0jynYGnkyRJ6rKg06Grag2wprnT4/MGE0mSJC3EqJ0OPUhtg3M3SvKmJIcneVI6XkFnwO5zlyeiJElSR1uLyyfpDM79LvBi4PXAesAzquqMwUaTJEnzMUmnQ7cVLvetqq0BkhwJXAXcq6quG3gySZI0L5N0VlHb4Nyb1j6oqj8BP7NokSRJw9LW4rJNkmubxwE2aOYDVFVtNNB0kiSplV1Ft1i/qm5q2UaSJGlZtHUVfX9ZUkiSpEWrAfwbVW0tLlmWFJIkadGmJmhwblvhctckB861sqre3+c8kiRJc2orXFYAG2LLiyRJI2ty2lvaC5fLquofliWJJElSC8e4SJI05jwd+hZ/neQ1wP2Bs4CPVtXNCznAy382Opd6eed6C4o+MH/3nY2HHQGAF904Ome6/9/brTfsCAC8aWo0bg7/nsvuOuwI0968xWXDjgDAsz+3/rAjAJAR+nvucV96y7AjAPCG094x7AgA7LP9nEMyl92nhx3gf7G2wuUQOlfP/RbwFOBBwKsHHUqSJM2fLS63eFDXvYo+Cvxg8JEkSdJCeK+iW3Tfq2g0+lkkSdLE8l5FkiSNObuKGlW1YrmCSJIktWlrcZEkSSNulO8t1G8WLpIkjTkH50qSJI0gW1wkSRpzkzQ41xYXSZI0NmxxkSRpzE3SGBcLF0mSxpxdRZIkSSPIFhdJksbcJF3HZcEtLknukmR07usuSZImRs/CJckjk5yc5PNJtk1yNnA2cEWSXZcnoiRJ6mWqqu/TqGprcTkcOBj4NPAN4MVVdXfgccC75npSklVJ1iRZc9Hvf9G3sJIkabK1FS7rVNVXq+o/gMur6nsAVfXjXk+qqtVVtUNV7XDfDe/dr6ySJGkWNYB/o6ptcO5U1+MbZqwb3a9KkqQJMspdO/3WVrhsk+RaIMAGzWOa+fUHmkySJGmGnoVLVa1YriCSJGlxRq1rJ8mdgGOALYCfA8+rqmtm2W5j4EhgKzo9OX9TVd/ttW8vQCdJkvrtIODrVfUA4OvN/Gw+AJxQVVsC2wDnte3YC9BJkjTmRnCMy+7Ajs3jo4CTgTd2b5BkIzpnKe8HUFU3Aje27dgWF0mSxtwInlW0SVVdBtD8f7dZtrkv8Gvg40lOT3Jkkj9r27GFiyRJuo3ua7I106oZ67+W5OxZpt3neYh1gO2AD1fVtsD/MHeX0q2eJEmSxtgguoqqajWwusf6J861LskVSTatqsuSbApcOctmFwMXV9X3m/n/xzwKF1tcJElSvx0L7Ns83hf40swNqupy4FdJHtgs2hk4t23HtrhIkjTmRu10aODdwGeTvAj4JfBcgCT3AI6sqqc2270S+FSS9YCLgP3bdmzhIknSmKuaat9oGVXVb+i0oMxcfinw1K75M4AdFrJvu4okSdLYsMVFkqQxNzV6XUUDY4uLJEkaGwNvcfn47qPT77b3F0ejTjvm9ZsOOwIAn3/Xte0bLZOjDth42BEA2OoffzDsCACc/fePGnaEabu8+9fDjgDAiS+4w7AjAJCNNhx2hGk3nfmrYUcAYJ/tDxx2BACOPu39w44wNDV6V84dmNH4TS5JkjQPjnGRJGnMTdIYFwsXSZLGnF1FjST3Wq4gkiRJbdpaXL5I5wZIkiRpRA3iXkWjqm1wbpYlhSRJ0jy0tbhsluSwuVZW1av6nEeSJC3QCN6raGDaCpcbgNOWI4gkSVqcSRqc21a4/KaqjlqWJJIkSS3axrjcONvCJI9O8sEB5JEkSQs0RfV9GlU9W1yq6pFrHyd5KLA38DzgZ8DnB5pMkiRphp6FS5K/APYE9gJ+AxwDpKp2WoZskiRpHhzjcosfA98CnlZVFwIkOWDgqSRJ0rx5HZdbPBu4HDgpyb8m2Rmv7SJJkoakZ+FSVV+oqj2ALYGTgQOATZJ8OMmTliGfJElqUVV9n0ZV272KPgFQVf9TVZ+qqt2AzYEzgIMGnk6SJKlLW1fRQ2YuqKqrq+ojVfWEuZ6UZFWSNUnWfPzsXy45pCRJmpunQ9/iDkm2ZY5xLVX1wzmWrwZWA1z3qt1G96uXJEljpfVeRcD7mL1wKWDOVhdJkrQ8RnlMSr+1FS4X9uoSkiRJw+fp0JIkSSOorcXljd0zSdYFtgIuqaorB5ZKkiTNW43wYNp+a2txeVaSBwMkuSPwI+Bo4PQkew06nCRJUre2wuWxVXVO83h/4IKq2hrYHnjDQJNJkqR5marq+zSq2rqKbux6vAvwHwBVdXnilf8lSRoFk3RWUVuLy2+T7NZcy+XRwAkASdYBNhh0OEmSpG5tLS4vBQ4D7g68pqoub5bvDHxlkMEkSdL8TNLg3J6FS1VdAOw6y/ITk/zlwFJJkiTNYinXcTmwbykkSdKiTdLdodu6inpxdK4kSSNglAuNfltKi8vkvEqSJGkk9GxxSXIdsxcowbOKJEkaCZPUktA2OHflcgWRJElqk3HoF0uyqqpWDzsHjE4Wc9zWqGQxx22NShZz3NaoZDGH5mtc7g69atgBuoxKFnPc1qhkMcdtjUoWc9zWqGQxh+ZlXAoXSZIkCxdJkjQ+xqVwGaX+xlHJYo7bGpUs5ritUclijtsalSzm0LyMxeBcSZIkGJ8WF0mSpOEVLknunuQzSX6a5Nwkxyf5iyQ3JDmjWXZ0knWb7XdMclzzeL8klWTnrv09s1n2nD5ke2aToXuaSvK3zTFe2bXt4Un2W+Rxft/8v0Wv/Sb5RJKfJflRkgua12Wzmfvpmt8vyeHN4wcmObn5Gs5L0rMZtMf7cvaM7d6W5HVd8+skuSrJu2Zst1uS05vs5yZ56SJep0ryvq751yV5W9f8qiQ/bqYfJHlMs/zAJB/t2u75SfpyV/Mkf2pe07OTfDnJxs3yte/lO7q2vUuSm9a+J/3S9T2/Zdeyhzfv90+S/DDJV5Js3ax7W5JLZnxfb9ynLGtfj3Oa9/rAJLdr1nV/djdJclzX98Px/Tj+jAy3ek+61v8oyadnLOv52Vpkjjt3vb6Xz3jNN2m+F17atf3K5vP2gGZ+3SRnJXnEPL7O/0iyWY/jrbeQ1yXJ/l3PvbHJcUaSd6fr50qz7ayfu0W8Xmvz/aj5nv2rxexnjn3/fpZlt/mZmOTJXV/375Oc3zw+unnOrT5rSb7frP9lkl93PXeLfmVXD4O4MdM8btwU4LvAy7qWPRR4LHB2M78C+Abw/GZ+R+C45vF+wJnAkV3PPwY4A3jOAPKuAk4B7gtcAVwIrNesOxzYb5H7/X3z/xa99gt8Yu3X1bx2BwAXdG37+xn73Q84vHl8IrB717qtl/K+dC1/G/C6rvmnAt8GfsotXZDrApcCmzfztwceuIjX6Q/Az4C7NPOvA97WPN4NOK1r3XbAL4G707nA4hnAo4GNm33ct0/fE7/venwU8Oau9/KnwOld6/+2yXF4n78vPwt8q+u12AT4OfBXXds8BnjGbO9Zn7N0vx53A74GvL2Z35FbPrsfAV7dte1DBpRh+j1p5v8SOAu4BPizruU9P1t9yDTzc/J3zXt28oztngd8tXn8JuAj8/w6PwUcONfxFvu6NOt+vvZz1czvxy0/V+b83C3xfXsycMogvie6lvX8mQicDOwwY9mtPmuzvSZOyzcNq8VlJ+Cmqjpi7YKqOgP4Vdf8n4AfAHP99fMt4OHNXycbAven88uhr5L8BfBW4IXAFPBr4OvAvn0+1Lz2Wx2HAJcDT5nHfjcFLu56/lk9tm19X3rYC/gAnR9ej2yWraRTPPym2dcfq+r8eexrppvpDJg7YJZ1bwReX1VXNcf4IZ0fzi+vqpvp/KL4IPBPwMeq6qJFHL/Nd7n19+kNwHlJdmjm96Dzg69vmu/5RwMvAvZsFr8COKqqvrN2u6r676r6Yj+P3aaqrqRT7L8iycybsc78fjxzQDFmvid7A58Evgo8fbYnLOKztRh7Aa8FNu9u2amqzwJTSd4AvIxO8TIf36Lzs2++Fvy6zGHOz90C9jGbjYBrlriPNgv5mTjXZ01DNKzCZSs61fqckqwPPAI4YY5Nis5fdU8GdgeO7WfAJsO6wL/T+Qvml12r3g28NsmKPh9yIfv9IbBl61ZwCPCNJP+Z5ICZzcQz9Hpf7tfVHHoGnR+uACTZANgZOA74NJ0fzlTV1XTel18k+XQ6XTWL/Z77IPD8JHecsfzBs2Re0yyn+SV+HvBEOsVLXzXv1c7c9vvvM8CeSTYH/kSn5amfngGcUFUXAFcn2Y7O1/zDlucd0PU+ntTnTNOaAvF2dFpfun0Q+GiSk5K8Ock9+n3sOd6TPei0yk5/f/Yw38/WQnPdk06LxA/oFLJ7zNjkNcB7gH9sPjtt+1uHToHV8xdv1/ZLfV269fzcLdAGzffjj4EjgXe0PWGJFvIzEWb/rGmIRnFw7v2aX4y/AX7Z8hfZZ+hUwHvS+eD12zuAc6rqM90Lq+pndFqD9u7nwRa435l/yd5md80+P06nOfg/6DTZfy/J7RcR76dV9dC1E3BE17rdgJOq6nrgc8Az1xZfVfViOj8sf0Cni+djizg2VXUtcDTwqnlsHpqvv/lraQc63VZ3Xcyx57BB1/fpnYD/mrH+BGAXOr8Mjunjcdfai873P83/t/ml0/TDn5fkA12LD+l6H3caQK5bRZi5oKpOpNPl+q90ioPTk/TrfZn1PUnyMODXVfULOq2a2yX584Xk7pM9uaXlbbb3bFfgMjp/QPSy9utcQ6eF86O9N+/b69Jm+nO3QDc0349b0nkNjp6lpa5vFvEzsfWzpuU1rMLlHGD7Odb9tPnFeH/gkUnmbL5s/nLZik4/6wX9DJhkR+DZdJrfZ3MwnebSfr+G893vtnRaEgBuSLJe17o7AVetnamqS6vqY1W1O51ul7l+MPZ6X3rZC3hikp/T+SvsznS6ndYe/6ymCX4XOq/pYh1Kp7n2z7qWncttM2/XLAd4O/BvwDvp/KXVLzc036f3BtZjRhN5Vd1I57V4LZ1irm+S3Bl4AnBk85q/ns5fzufQ+drXZngE8BZgZivVwCW5L52Wpitnrquqq6vq36vqhcCpwOP6dNi53pO9gC2b1+qndLojen0fdn+2+mkvYL8mx7HANrllQO496BTlDweemuQhPfZzQ1fx+crme62Xfr0u3do+d4tSVd8F7kJ//8iY7Tjz+pk412dtkIWV2g2rcPkGcPskL1m7oKn+7712vqouAw6iva/3TcD/6We45q+OjwP7VNV1s21TVT+m8yHdrZ/HbttvOl5Fp592bTfaKcALmvUb0Bnod1Izv2tuOTPr7nSKikvmOHzr+zJLno3oDAC9V1VtUVVb0PnBuFeSDZsCcK2HAr+Ya19tmubzz9IpXtb6J+A9zQ8YkjyUzoC5D6VzNs1f02l+Xw3cO8kuiz3+HJl+R+cXzuvWvs5d3ge8sap+089jAs8Bjq6qezev+T3pDDz+Kp1fjN1nZdyhz8du1bSgHEFn0GLNWPeEJHdoHq8E7ken1aBvZrwntweeS2cQ8Nrvz92ZvYVqts9WXyR5IJ3Br5t15XgXt4yZOAQ4uKouBg4EPtjvX46LfV3mMOfnbikZ0zlrZwXNuLhBWODPxLk+a4s6g0r9sc4wDlpVleSZwKFJDqJz1sjP6fTxdvsi8LYkj+2xr/8cQMSX0emb//CMnx0zu6PeCZw+gOPPtt/3JnkLnV9E3wN26vpL69XAR5ofuqHzQftms+5JwAeS/KGZf31VXT7bQRfwvnR7FvCNqvpj17Iv0fnBdiDwhiQfoTNg9X/o/HBbivfR1QpWVcemM8jxO0kKuI5OEXc5nabgA6rqDwBJ/o5OM/RD5/FX6rxV1elJfkTnl9C3upafQ6cVpN/2ojMeqtvn6HQx7kHnF8pmdFo7rgL+oWu7A5K8oGv+GVX18z5kWtsdsS6dv2A/Cbx/lu22Bw5PcjOdP5yOrKpT+3D8W+l6T54HXFJV3b+Yvgk8KMmmzXyvz1a/7AV8YcayzwGfSfI94F40XT5V9eXmj4d96Ax47Zv5vi7NH4699jPr567teXNY+70DnZ9f+1bn5Ix+uEOSi7vm3w9szjx/JtL7s/at226u5eCVcyVJ0tgYxcG5kiRJs7JwkSRJY8PCRZIkjQ0LF0mSNDYsXCRJ0tiwcJEkSWPDwkWSJI0NCxdJkjQ2/j+5KhSZihhnAgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (10,8))\n",
    "sns.heatmap(X.corr())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a12faa96",
   "metadata": {},
   "source": [
    "#### （3）使用VIF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "816c7077",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import library for VIF (方差扩大因子)\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    " \n",
    "def vif(X):\n",
    " \n",
    "    # Calculating VIF\n",
    "    vif = pd.DataFrame()\n",
    "    vif[\"variables\"] = X.columns\n",
    "    vif[\"VIF\"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]\n",
    " \n",
    "    return(vif)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "391f982e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.100373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.844013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.485758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.152952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>77.948283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.386850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>14.699652</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.167725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>61.227274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>PTRATIO</td>\n",
       "      <td>85.029547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>B</td>\n",
       "      <td>20.104943</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>11.102025</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.100373\n",
       "1         ZN   2.844013\n",
       "2      INDUS  14.485758\n",
       "3       CHAS   1.152952\n",
       "4        NOX  73.894947\n",
       "5         RM  77.948283\n",
       "6        AGE  21.386850\n",
       "7        DIS  14.699652\n",
       "8        RAD  15.167725\n",
       "9        TAX  61.227274\n",
       "10   PTRATIO  85.029547\n",
       "11         B  20.104943\n",
       "12     LSTAT  11.102025"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a5e2fd5",
   "metadata": {},
   "source": [
    "从中找寻vif最大的再从X中去除，每次去除一个都需要重新再计算vif。以此类推，直到剩余的各特征vif在标准范围之内即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "62434cd3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.099345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.451624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>14.275283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.142167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NOX</td>\n",
       "      <td>73.894171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RM</td>\n",
       "      <td>60.598846</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>AGE</td>\n",
       "      <td>21.361234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DIS</td>\n",
       "      <td>12.221605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.159162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TAX</td>\n",
       "      <td>59.301541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>B</td>\n",
       "      <td>18.614751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>10.138324</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.099345\n",
       "1         ZN   2.451624\n",
       "2      INDUS  14.275283\n",
       "3       CHAS   1.142167\n",
       "4        NOX  73.894171\n",
       "5         RM  60.598846\n",
       "6        AGE  21.361234\n",
       "7        DIS  12.221605\n",
       "8        RAD  15.159162\n",
       "9        TAX  59.301541\n",
       "10         B  18.614751\n",
       "11     LSTAT  10.138324"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('PTRATIO',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "2d911854",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.097537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.449661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>13.150904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.138277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RM</td>\n",
       "      <td>41.406746</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>AGE</td>\n",
       "      <td>19.889623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>DIS</td>\n",
       "      <td>12.032952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>RAD</td>\n",
       "      <td>15.155012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>TAX</td>\n",
       "      <td>57.720347</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>B</td>\n",
       "      <td>18.396072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>9.207840</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   variables        VIF\n",
       "0       CRIM   2.097537\n",
       "1         ZN   2.449661\n",
       "2      INDUS  13.150904\n",
       "3       CHAS   1.138277\n",
       "4         RM  41.406746\n",
       "5        AGE  19.889623\n",
       "6        DIS  12.032952\n",
       "7        RAD  15.155012\n",
       "8        TAX  57.720347\n",
       "9          B  18.396072\n",
       "10     LSTAT   9.207840"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('NOX',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "11556cf7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.097466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.375219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>9.290080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.118613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RM</td>\n",
       "      <td>39.069063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>AGE</td>\n",
       "      <td>19.780944</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>DIS</td>\n",
       "      <td>11.817803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.965633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>B</td>\n",
       "      <td>18.289917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>9.046248</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables        VIF\n",
       "0      CRIM   2.097466\n",
       "1        ZN   2.375219\n",
       "2     INDUS   9.290080\n",
       "3      CHAS   1.118613\n",
       "4        RM  39.069063\n",
       "5       AGE  19.780944\n",
       "6       DIS  11.817803\n",
       "7       RAD   4.965633\n",
       "8         B  18.289917\n",
       "9     LSTAT   9.046248"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('TAX',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "e20c0850",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.095367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.334763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>9.016142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.116229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AGE</td>\n",
       "      <td>14.000758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>DIS</td>\n",
       "      <td>8.447694</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.771767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>B</td>\n",
       "      <td>13.537020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>8.358925</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables        VIF\n",
       "0      CRIM   2.095367\n",
       "1        ZN   2.334763\n",
       "2     INDUS   9.016142\n",
       "3      CHAS   1.116229\n",
       "4       AGE  14.000758\n",
       "5       DIS   8.447694\n",
       "6       RAD   4.771767\n",
       "7         B  13.537020\n",
       "8     LSTAT   8.358925"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('RM',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "270d6103",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.095211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.313889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>8.205465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.106350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>8.209371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.689836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>B</td>\n",
       "      <td>10.074224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>6.856889</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables        VIF\n",
       "0      CRIM   2.095211\n",
       "1        ZN   2.313889\n",
       "2     INDUS   8.205465\n",
       "3      CHAS   1.106350\n",
       "4       DIS   8.209371\n",
       "5       RAD   4.689836\n",
       "6         B  10.074224\n",
       "7     LSTAT   6.856889"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('AGE',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "d4942206",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variables</th>\n",
       "      <th>VIF</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CRIM</td>\n",
       "      <td>2.067135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ZN</td>\n",
       "      <td>2.299351</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>INDUS</td>\n",
       "      <td>6.900077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHAS</td>\n",
       "      <td>1.086574</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DIS</td>\n",
       "      <td>3.968908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RAD</td>\n",
       "      <td>4.678181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>LSTAT</td>\n",
       "      <td>6.726973</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  variables       VIF\n",
       "0      CRIM  2.067135\n",
       "1        ZN  2.299351\n",
       "2     INDUS  6.900077\n",
       "3      CHAS  1.086574\n",
       "4       DIS  3.968908\n",
       "5       RAD  4.678181\n",
       "6     LSTAT  6.726973"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X.drop('B',axis=1)\n",
    "vif(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fda56bf",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
